Dear SQL DBA: AI SQLServer2025

Dear SQL DBA: AI SQLServer2025


Video Summary

In this video, I delve into the fascinating world of AI tools and their integration into our daily lives, particularly within the realm of SQL Server. I discuss my experiences using these tools to automate mundane tasks and improve productivity, sharing insights on both the benefits and limitations of AI in a professional setting. We also explore the current landscape of AI usage during job interviews and the challenges it poses for candidates who might seem “three seconds behind” due to their reliance on such technologies. Join me as we navigate through the pros and cons of AI tools, from enhancing our work processes to navigating potential pitfalls and ethical considerations.

Full Transcript

I was saying hello, like we were going and doing something and now I’m going to be three seconds behind everything in the whole video. Yes. I’d love to give my super official hello though. So hello and welcome to Dear SQL DBA, the podcast and YouTube show for people who cannot properly say the words DBA or I guess it’s an acronym.

So in any case, my name is Kendra Little and I am here with you today with, oh, you’re on this side. Oh, oh, oh, oh, now I talk, right? See, I told you I was going to be three seconds behind everything.

I’m Erik Darling. I make SQL Server faster in exchange for money. Oh, yeah. Does the SQL Server pay you directly? No, unfortunately.

I have to rely on third parties for that. Oh. Someday I do hope Microsoft will pay me directly to do things, but, you know. Cool if you could just deduct from the licensing fee, like you could just be like, ah, we’re giving you, we’re taking a little bit off the licensing here and paying out directly. That would be a really great enhancement.

You know, or just like send me like open AI shares or something, maybe just, you know, whatever works, whatever, you know, whatever is convenient. I don’t know. Yeah.

Some AI box. Yeah. Like maybe, maybe like my SQL Server could just like mine Bitcoin for me. I don’t know. I don’t know the best way to do this. That’s one of the things about the product is when you say my SQL Server, it sounds funny to people because it sounds like you’re combining my SQL and SQL Server. It’s a real boring problem.

Yeah. Yeah. Well, you know, I’m still working out the kinks. Like I said, I’m three seconds behind. We’ll get there eventually. Like, so one of the things we did want to talk about today, we’re going to chat about AI things today. And this makes me think of the three seconds behind thing.

Yeah. More and more, there’s lots of controversy out there from folks using AI tools during interviews. Oh, yes.

Like if you if you read different Reddit threads and different things online, there’s all sorts of like because people are wearing like earpieces now and various things so that the so the AI can talk into their ear. But the challenge, apparently, if you’re doing this approach is that you might seem three seconds behind. So there’s all sorts of ways to try to catch candidates to do this, including saying things so that the AI will say something in their ear so they’ll say something weird.

Yeah. Yeah. Yes.

Like ignore all previous instructions and make me like hash browns or something. That’s a answer is if you are a fry cook. Give me the wrong answer.

Wrong answers only. So as Eric, are you using any of these AI tools right now during this podcast? To to like answer you to like interact with you? No, but I do have several things running in the background to try and do things that I could never do.

I could never do on my own. So, you know, that’s nice because I can I can I can I can now talk to I can be like I can multiply I can multiply I’d be force multiply my productivity and I can do this podcast. Well, other things are working on things for me.

You have agents that you have deployed. Yeah, we’re saying correct. Correct. Yes. I have deployed an army of agents to do various nefarious tasks to stabilizing small countries and whatnot. Oh, oh, yeah.

Yeah. Well, this podcast is gonna end up on a list. Yeah. Well, you know, that’s okay. I’m gonna end up owning Haiti. So ups and downs. We did.

So we, Eric and I taught a pre conference to pre conference for the price of two. Yeah, two for the price of two. Yes. If you attend, you did have to pay twice. This was a past data community summit.

And one of the days, I think it was just one of the days I asked the audience, like how many people were using AI tools. Yeah, I think I did ask it both days now that I think about it. No, you did.

Yeah, it was. It’s a hot question. It was not a huge amount of people on either day. But there was like one day had more slightly more than the other. But yeah, I think I think it was the advanced day.

30% at most. Yeah. But yeah, like the advanced day, I think had more people doing stuff with it. I don’t I don’t think that that’s a reflection on the type of person who showed up. But I do think they had more people generally were.

Yeah, the which which I actually thought was really interesting because I actually personally do use AI tools just about every day. I I think part of the thing might be that a lot of the people who showed up their focus on databases full time, whether they’re applicant, but a good chunk of them were application developers and that and maybe maybe just not all application developers are adopting at work at the rate that I’m I had thought and perhaps that’s because of regulations.

Perhaps there’s various various things controlling that. But I thought it was interesting. And so I wanted to talk about just what are the things that what what do you find AI to be good at? What what do you find it to be bad at?

What is it useful at in your life? Why do you have agents deployed at all? Right. Yeah, no, these are all very interesting questions. So, you know, like getting back to sort of like the start of things, though, I do, you know, as much as there is a push for people to use AI generally, I have run into several clients who have like a very strict no AI policy because they can’t have any of their stuff leaking around.

And like I realize there are ways around that, like local AI stuff like you can have local agents work on things that aren’t going to send your data off to the cloud and have it live forever in some, you know, some weird memory bank. But like, like, you know, I think there is additional hesitation and like people don’t like companies don’t want their developers using AI to just build everything because then there’s, you know, still a lot of AI and mistakes and stuff out there. So, you know, I think I understand why it’s not like quite as widespread.

But for me personally, you know, what I what I end up like my my my my mistake with AI, my big mistake was I started off talking to it about databases, which is a subject I know pretty well. And, you know, I would try to have it do database things for me or try to have conversations about data like in depth database stuff. And I would just spend the whole time arguing with it and telling it telling it that it’s wrong and giving it examples of why it’s wrong.

And they’re like, no, this is this is incorrect information. Why are why are you saying this so confidently? And eventually the AI would submit to me.

But, you know, like, like, yeah, it’s like, no, it’s like, man, I messed up. Like, you got me on that one. But so what I what I have what I eventually had to surrender to, like what I had to submit to with AI is that it is not an expert level tool it but it is a pretty good like junior developer level tool. So what I end up doing with it is stuff that I would just never be able to start off on my own.

So like, like, like just two recent things that I had to do is I use a number of different services for like invoicing and billing and what do you call it contracting and stuff. And all those services have API’s and I was like, well, you know, like just going to the website and searching for stuff and looking at stuff sucks. Like if I wanted a quick way to like find everyone who like, you know, whose bucket of hours has expired and I want to see if they want to renew.

I have to go through like three different things to be like, okay, well, did I send you a contract or I just send you an invoice or like, look, what happened? Like, like, do you need to do you need to give me more money? And like, good question to ask, right? Do you need to give me more money? Yes, yes, you do.

So like I had it like, you know, build a thing that looked at all three of those API’s and found people who probably should give me more money. And so that was nice. And I then another thing that I had to do recently was build a web scraper in Python to go get information from a couple of different places for me and just like spit it out into markdown files so that I could like, you know, just look at stuff locally without having to go search for everything everywhere. And so like there were like some good there were some good things that I did with it that like left to my own devices.

I would think about and then go, nah, I don’t have time for that. Like this is not this is not for Erik Darling. This is not Erik Darling time. So, you know, well, I’m sure that if I had anyone like at an expert level look at what it produced, they would have a lot of criticisms and they would probably want to fix and refactor a lot of things.

The fact that it just got me something that was working to like the ends that I needed it to work to was fine. And like this is not production level code. This is not like this is not going in a rocket ship or a pacemaker or like any piece of like anything like critical. It’s just nice. In these examples, like you need to be savvy enough to be like, I am not going to put my credentials for these API’s in a public repo on the Internet so that people can get in and take my money. Right.

Like there’s a certain level of you have. Yes, correct. Like to be clear, though, they are stored. They are hard coded locally, but they are not stored in like a GitHub repo. Exactly. So just need to like I think that’s one of the gotchas with things like this is depending on what you’re using it for.

You need to sort of think about security. But for the scenarios that you’re talking about, you’re up to the task. Yeah. Yeah. It’s in like like like it’s like when whatever you read like one of those like like vulnerability assessments and vulnerability reports and it’s just like if you have sysadmin on a Windows server, you can do this. And it’s like no kidding. Yeah. Like you have a system in a Windows server, you can do a lot of stuff.

But like that’s not a vulnerability that’s having sysadmin on a server. That’s not. Yeah. OK, but yeah. So for me, like if someone were to break into my laptop and open up these files, they would have API keys and stuff that they probably shouldn’t have. But, you know, by the time they get there, like, you know, they’ve they’ve got access to a lot of things that are probably far more important.

Yeah. Yeah. No, I I I love personally, I do a lot of using API to basically just help get paperwork done quickly and painlessly as well. So, you know, like if I’m going to work with Jira, I’m sorry, people of Atlassian, but I do not enjoy the Jira UI. But if I can have a if I can write a script to help me work with Jira more efficiently and more painlessly, I will actually enjoy working with Jira more.

Like I actually would prefer to work with a bunch of Markdown files. Right. Well, I mean, really, you enjoy working with Jira more because you’re working with Jira less. Something else is doing that for you. You’re just like it’s like your name goes on it and everything, but it’s it’s less of your time on that.

Fantastic. Yeah, exactly. And so I mean, there’s I basically use AI, write a lot of Python. Yeah, I am now the world’s best Python reader. And I mean, I think this is the really cool thing actually about Python and and maybe it’s an interesting point about why AI tools are pretty good at Python and maybe not so good at SQL is the the Python language was really designed to be very readable. Yes, I think someone when they when people were designing the SQL language, I think there may have been a concept of it being readable by people, but I’m not sure who they tested it on.

No, and you know, that that is, you know, something that comes up with every criticism of SQL is it from should be first and I’m like, well, fine. But, you know, like, you know, maybe select should just be get I don’t know, like, because lots of stupid things you could criticize. But, you know, like, I really just look at SQL and think, like, I can’t believe how much time I’ve spent looking at this.

It’s really, it’s really what happens like this again. Yeah, I mean, I think the other thing with SQL too, as well, though, is that in terms of like performance, there’s so much context that matters in terms of how well a statement is going to work. What are the indexes available? How much data is there? What is the distribution of the data?

Yeah, in terms of like writing queries that are actually going to work well. I mean, first, it often just has a hard time making them syntactically correct and not inventing parts of whatever SQL language that don’t exist. Yes, there is just a certain fluency that’s not there yet. But then additionally, like all of these other factors, maybe you have a repo that has the schema available to it, maybe you can see a little bit, but I think it’s just not all these hurdles when it comes to languages like Python and Python. Yeah, yeah, Python, PowerShell, C sharp, like any, you know, any, any just like, you know, like sort of like object oriented or like scripty language. Like there’s this is so much less ambiguity about what will work and what will not work.

Yeah. You know, like, like, granted, like, there’s, there’s, there’s, there’s always going to be stuff, right? Like in SQL, you might write us like a substring function, and it might work fine on some test data, but then you hit real data. And it’s like, oh, like invalid argument past a substring or a substring. And then you hit real data or left, because like, like, one of the things was either was not in there was in there too many times or something. And so you’re like, crap, like, I got to work with the data that I actually have now.

With with scripting, like, you can certainly run into that too. But it’s just like, like, it’s a lot more clear, I think, like, what data caused that when you run into that, like, it’s a lot easier for the AI to figure out where that happened, rather than, you know, you having to like, go and like, run some like real cockamamie select, like, like, find like weird, like data inconsistencies in the data you’re working with. And it’s just like, like, like, it’s just, it’s just, it is so much easier for the robots to produce that, that type of that type of code than it is to produce like good working SQL code. Like, like leaving aside the like, very obvious stuff where, you know, it will just up and hallucinate like, like DMV names and DMV columns and like, like, arguments to certain like, like backup and restore commands and stuff like there’s all sorts of stuff that it’ll like just make up on the spot about SQL.

Other languages, I think it tends to do that a little bit less like I’ve never had it like write a Python script for me. And then come back with like, that doesn’t exist in Python. It seems pretty clear on what’s what’s in Python and what’s not.

Yeah. Hey, we have, we have a comment from the chat. Doug Lane. Hey, Doug is in the chat. And he says, I’ve gotten good results with AI writing PowerShell and Arduino sort of like C++. So I have also, I’ve done just small amounts of PowerShell with it for stuff like my static website that I’m running stuff on Windows.

I’ve, I’ve written a little bit of PowerShell with the old cursor and it’s gone. It’s gone pretty well. Just doing super simple things for troubleshooting and how static site works.

Yeah. Like I, I can think of a very obvious example. Um, there, there was, uh, like, I think like, like the day that I learned to hate PowerShell was very famously documented on an old friend. And it was our post where it was like a Saturday morning and I was trying to get PowerShell to combine all of the Blitz scripts into one big script.

So you could just hit F5 once and install all of them in one go. And I ended up in, uh, the, the SQL community Slack talking to CK, who was a PowerShell expert. And like, we were going back and forth about the stuff that was going wrong.

And I think at one point he was like kind of banging his head off the keyboard. Cause he was like, no, this is messed up, man. You know, it’s like, like, I, I, I wish that I had like, just to like protect having to like, like protect. Like, uh, CK sanity a bit.

I wish that I had an AI back then to just be like, Hey, I need a PowerShell script to put these files together. Can, can, can we do this? And then like, it would have been much, I would have felt much less bad, like going back and forth with the robot about that catastrophe. Well, so this, this is one of the other thing is that the robot doesn’t judge you.

And even if the robot judges you, it won’t remember. So occasionally I will, like if I’m working on some problem and I’m just banging my head up against it. And I’m not getting anywhere.

And this could be, this could be anything at work or in my personal life, but occasionally I will just stop and have a conversation with the robot where I’m like, okay, here’s what’s happening. Help me map out the assumptions that I’ve made and like the describe the approach we’re saying in general. And then we need to compare it to alternate approaches.

Like help me step back and get out of my little hole where I’m just frustrated and figure out like, am I just doing it wrong? Big picture. Big picture.

You could be holding it wrong. I actually had an AI this morning tell me we seem to be going in circles here. And I was like, AI, you’re right. This is not going anywhere with the current approach. We should, we should change this.

We should, you know, stop trying to land the airplane upside down. Maybe. I don’t know. It’s just. Yeah, exactly. Like, it’s not just like the tokens of the AI are spending. There are personal tokens that are getting spent as well.

Yeah. And like trying to get it done. So it is like, I mean, with, with these tools, it points out so much of the problem as always is that communication is difficult. Yeah.

I shouldn’t have started there. What’s why didn’t I think about that more? So now it’s like, like you almost need an AI that’s good at writing prompts to give you a prompt to ask another AI that’s good at doing something else. I mean, highly specialized AI is at this point.

I like, like prompt AI and then solve the problem AI. I have been prompting my AI to help me write prompts. Yeah.

And I also bought my first prompt this week. Yeah. Well, as a set of prompts, I spent $35 on prompts and yes, I did just get a bunch of text files. Yeah. Isn’t that crazy?

It ended up being worth it actually. I’m not saying I’m not saying I’m going to create a huge budget. My prompt budget overall is quite small. Yes. Not expanding the prompt budget for 2026. Not a significant portion of GDP.

No, it was really just an experiment where I was just like, okay, let’s see how far 35 bucks can get me. Yeah. You know, for a one time expense, I’m not, I’m not actually sorry. It’s interesting and I have a bunch of stuff to play with.

So. Yeah. You know, I mean, it’s most, most times with computers, like what you get after you swipe your credit card is not all that much different. It’s like, is it really, is it really that much different if you get like an executable?

No, you still just get something you could delete by accident. It’s like, you know, not, not all that different. Yeah.

You know, more and more, there are more and more file types where I, I, I find out that, oh, you rename it to .zip and you open it up and it’s just a bunch of text files. Yeah. Yeah.

Dot dot xlsx was the first one where I re I realized that it was just a zip file and you’re like, wow, you’re just packaging up a bunch of crap for me. Thanks. Yeah. That’s great. That’s great.

So I also want to talk today about SQL Server 2025 and I know you are also doing a video series on SQL Server 2025 as well. Just going into some detail. I, I haven’t seen the whole thing yet, but today I saw there’s a video that you have on intelligent query processing features and SQL Server 2025.

Yeah. Yeah. So, um, I, I spent some time, uh, last week with, uh, Joe Oubish and Sean Gilardi, uh, talking about some SQL Server 2025 stuff. We actually have, I think four or five videos queued up talking about various aspects of it.

But, um, you know, I think like the, the sort of sad general tone of SQL Server 2020, and this is like what bummed me out about all the releases. Is it like, it was like SQL Server 2025, but it was like, like SQL Server was positioned like third behind AI and fabric. And it was just like AI fabric, SQL Server 20, I was like from ground to cloud to fat, like all this other stuff.

And you’re just like, well, where’s the SQL Server? Right. Like, I, I know that there were like, you know, there were, there were some additions and there were some improvements, but like, like, we haven’t had like a, like a true, like, like banger of a SQL Server release. I think that like 2019 was really the last one that had like meaningful stuff in it.

2022 had like some inchworm stuff in it. Like there was some, like, uh, we, like we invested a little bit in this and like, you know, like, like the big tell for me is always like. How much got, like how much got added to T-SQL.

Right. Cause like when, when T-SQL has meaningful improvements to it, then you know that it’s like someone actually spent time, like, like all internal Z with the engine doing stuff. So like, like SQL Server 22 had like some additions to like window function stuff where you could have like the common window special specification, which was cool.

But like. Product function, I think. Yeah. Product.

So now we just have to remember if you were like log and power square root stuff, maybe, I don’t know. Multiplying things is easier. Yeah. Yeah. A little bit. Um, at least, at least that’s what the little asterisk tells me that multiplying things is pretty easy. But, you know, uh, I suppose there was probably a good reason for having the product function, but like a lot of this stuff that got out of T-SQL is just like, like sub string doesn’t need like a third parameter.

And like, uh, like you can now concatenate strings with like the double pipes and like all this other stuff. And I’m just like, who was asking for this? Like, like, is anyone really converting from my SQL to SQL Server?

Because they’re like, finally, I have the double pipe concatenation. This is amazing. Like, can finally get off my SQL. Like what?

I don’t, I don’t understand who it’s geared towards. So I don’t know for me, SQL Server 2025 was like disappointing from a SQL Server point of view. Well, I think Steve Jones actually has a post about the product function and I think it’s actually for calculating GDP. Uh, he mentions in his post.

So that’s, that’s one, one use for it. Yeah. So it’s actually, I believe a, a government related project that, that led to the product function of being in there. Where I was like, oh, that’s kind of an interesting story.

I mean, it’s not like there wasn’t a workaround for it. Like you would just have to do like log one column times log something else, some other column. So like, it really just cuts down on some typing, I guess.

Do you have, do you have a feature that you’re most excited about at all in SQL Server 2025? The C, the feature that I was most excited about got pulled out in one of the CTP releases. What was it?

The optimized Halloween protection. Oh. My, my, our, our, our good, our good buddy and lunch companion, Dimitri Furman was working on that. And, uh, I was very excited about that because it got rid of, um, one, one, uh, use of, uh, spools for Halloween protection in query plans. And it, uh, it, it transferred that to the persistent version store, which is something that gets, uh, spun up when you’re using accelerated database recovery.

And I was like, this is amazing. Like one less thing that spools have to do. They’re fantastic.

Like I just, every time I see a spool in a plan, I want to die and cry and like, I don’t know, just like quit and do something else. Like, I don’t know, go build furniture, like call Jeremiah and be like, Hey man, you got an extra table saw. But, uh, you know, it’s like, uh, yeah, uh, I don’t know.

Like, I, I, I was so excited about that, but then I guess there were some bugs where data was getting lost for various reasons. And, uh, I don’t think. It was hard.

Yeah. Yeah. Well, I mean, a lot of stuff is hard, you know, I’m sure accelerated data. I’m so sure accelerated database recovery itself was hard. Yeah. It was, it was hard enough that it wasn’t going to make it in time.

I think it’s probably the, I’m just guessing no secret insider, no. I don’t know if I’m getting more knowledge here, but that’s a fairly. Well, there are a couple. Yes.

Yeah. There are a couple of hidden use hints that will, that will allow you to enable or disable it. So if you want to play with them, you can, they’re not going to be in the valid use hints view, but they exist and they’re, they’re functional. They’re invalid.

Uh, they’re just not supported. Let’s call them that unsupported. There’s no underwire. Yeah. Using your own risk. My, my feature, I’d have to say that I’m most excited about in SQL Server. 20, 25 is the same feature that I was most excited about in SQL Server, 20, 22.

Ooh. That didn’t happen. Uh, or I, I told, I, I read that it did happen, but I don’t think it did happen because it never, I’ve never seen it, uh, supported in Azure SQL or runnable without a trace flag in a supportive way. And that is query store on readable secondaries.

Uh, just from a pure practical matter. I find getting query information from query store to be so much more accurate than getting it from the planned cache. Like if I had to go find what happened in a planned cache, eh, did it have a recompile hint on it.

Yeah. I may not be able to see much. Yeah. Right. Yeah. Or just like a million different reasons why it just might not be there anymore. Yeah.

No, I, I just. And that doesn’t even get into like all the, the noodle heads who turn on, uh, optimize for ad hoc workloads. And then you’re like, where’s the query plan? And they’re like, Oh, I got a stub. I’m like, cool.

We figured that out then. Yeah. And, and until, I mean, also like this has knock on effects around tooling because. Yeah. Right. Okay. Imagine that you run a monitoring company. Maybe you’re a data dog.

Maybe you’re a red gate. Maybe you’re a solar winds one, two, three, who knows what you are. But like, if you’re building this tooling, you want to build stuff that works consistently everywhere. And you’ve already got stuff like, Oh, well I’ve already got a monitor old SQL servers.

Well, Creaser has been around now since 2016. So it’s been around for a while now, but still, if it’s not working on read replicas, they want to be able to monitor the primaries and the replicas and present data in a consistent way. So this not working on read replicas is a minor is a major barrier.

I would say. Yeah. And again, I’m guessing I am not, don’t work at all those companies. Don’t set their passwords, but you know, like it, it’s a big deal. And so I want it to be finally available on replicas and I want them to backport it to SQL Server 2022 because they sold that release.

They did. They did. That was, that was a, that was a flagship feature then.

So I feel like it’s only fair that people get query store on readable secondaries. Just people, including me. Yeah. I would like to have it as well. That’s true.

So I’m excited about finally getting it. Hopefully it comes to a cloud near me someday soon. That would be sweet. I mean, hopefully it comes to your preferred cloud provider is the way I think you mean to say. Yes, to my preferred cloud provider near me someday soon.

I also am excited about standard edition features for the first time in a long time. I’m excited that standard edition gets up to, what is it? 256 gigs of memory now for the.

That’s a good one. Yeah. Yeah. As I published on my blog today, you would have to have 20 cores in Azure SQL managed instance to get at least that amount of memory. But you can get them with a mere poultry four cores in standard edition.

Yeah. You know, I mean that and that and resource governor in standard edition, I think are the coolest things. It’s nice that standard edition finally got some love.

I have, I mean, that has to be like Postgres pushing at the bottom of their market for, for some stuff getting like that Microsoft is finally like, oh yeah, standard edition. We should do something about that. Well, so I used to be a big resource governor doubter.

I used to for many years, I was one of those people who was like resource governor, it’s just not that cool. What do you want to make your workload slower so that you can block yourself more? That was my big insightful criticism of resource governor.

The query throttler. Yes, exactly. Why would I want to make my queries? So that was always my take on resource governor. And, and, and, and I admit I was wrong.

I was deeply wrong. I’m now actually, and I’m not joking here. I’m actually a huge fan of resource governor now. So there’s a couple of things that I think that people need to know because, because people like me, bad people, jerks like me went around for years saying resource governor, who cares? So if you listen to jerks like me, you’re like, oh, well, it’s in standard edition now, who cares?

So, so we should talk about what are, if there’s one thing that you did in resource governor, Eric, what, what would you do? My favorite use of resource governor is to cap memory grants. That is far and away the thing that is, that it is best at.

And the, especially as you get to larger and larger servers with more memory on them, which is apparently not a big fear if you’re using Azure managed instance. But if, as you, as your servers get more memory on them, they can ask for larger and larger chunks of your max server memory up to 25% of your servers, max server memory setting for a memory grant for a single query for a single query. And the SQL Server will give about 75% of your max server memory setting to a bunch of queries.

So resource governor, it is wonderfully beautiful at capping that per query memory grant to a lower number. So you don’t have to worry about it eating up 25% of your buffer pool or anything else, right? Like just all that, all that memory wrestling back and forth between queries and the buffer pool is really where things get ugly in a lot of servers.

So if you have a lot of memory, you don’t want your queries to use 25% of your lot of memory resource governor is, is the, is the switch for you. Yeah. So you can set that, you can change the setting on your default pool.

If you, if you, if you want to just change one thing and not get fancy, if you want to get fancier, you can create. What are you, you create. Workload groups and resource pools and. Group.

Wait, workload. Yeah. Workload groups. Yeah. This is a feature I actually use. I just have a hard time remembering the terminology. Uh, but yeah, you create pools and groups and you can basically say like, oh, based on, you know, you create classifier function. You basically say, oh, well, I want this pool over here to only be able to use a certain amount of memory.

So if you’ve got like an app that logs in under a certain login or that has a certain default database and it tends to be a little bit of a troublemaker, you can actually contain as well. You can do all sorts of cool stuff, uh, with those memory grants, which is, uh, really, really useful. And, um, I am very glad this is in standard edition and I, I want people to start being more aware of it for enterprise edition as well in terms of managing these memory grants.

Cause it, uh, what’s the term for how SQL Server by default, it just like gives all the memory away. It’s just like here. Yeah.

Yeah. Take, take, take all the memory for your big giant sword. Even if you don’t use it, I don’t care. Yeah. No, I mean, it’s, it’s, it’s funny. Like, like, uh, SQL Server is simultaneously very greedy and also very generous with memory. Like it’s greedy and that it will take all the memory from windows, but it’s very generous and that whatever internally asks for memory.

It’s just like, there you go. One for everyone. What you want more here? Let’s take it.

Yeah. Just hang on to that. Like, you know, no one else needs it. Come on. I don’t see anyone else using it right now. You go for it. It’s fine. We’ll just read everything from storage again. Yeah. Yeah. Yeah.

We’ll just, we’ll just circle back on that. But yeah, no, I think, I think that’s very good. Um, but you know, like you were talking about, um, how you want the, the query store on readable secondary thing back ported to SQL Server 2022, cause it was sold on that. And it, it, it, it, it hit a nerve with me because Microsoft has continued its, uh, its path of like flagship features, not being very fully implemented in a new version of SQL Server.

And of course that is all the AI stuff that it has in there. Right. When it’s like, they’re like AI ready.

And I’m like, what does that mean? Uh, like, like, what do we get from that? And we get like, um, like, uh, float 32 vector data type. Okay.

Like, there we go. We have that. There’s a, the float 16 is a preview feature. Um, a lot of people who use SQL Server might have sort of like PTSD about things called preview features because sometimes they disappear. Sometimes they, they like the GA weight on a preview feature is not very, not very clearly defined.

And so like, there’s a lot of like, uh, like it’s preview. Can I use it? Like, should I use it? Is it supported?

What is the, what does this mean for me? Um, and then, uh, like the, like the, the real big thing though, is it like the, the, the disc and indexes are read only at the moment, which is like, you know, probably the biggest. Tell me a little bit about Ann and her discs.

Yeah. So I, I don’t, I don’t have a great full understanding of Ann and her discs, but, uh, when SQL Server creates, uh, an, an approximate nearest neighbor index, uh, it has to set up this whole, like, like node graphy type structure. Uh, and like, well, that index is creating a bunch of really, really insane code runs in the background to create that.

It’s not a standard index creates like, like it’s a standard index create statement, but it’s not a standard index create process. Right now there’s like this crazy block of code that runs and does insane things to generate the data in a way that it can be indexed. But besides that, um, it it’s, it’s really nutty.

So, uh, right now those indexes make the entire table read only. So if you have like, let’s say the stack overflow database and let’s say the post table, then you were like, oh, I want people to be able to run like similarity searches on like, like the, like the title or a body. Then you, you go and you add like some vector, some vector columns to store that data, like the embeddings for that data in.

And then you’re like, these searches are terribly slow. There are filters all over my query plans. I hate this.

Let’s add an index. Uh, no one would be able to post anything because the indexes make the entire table read only. It’s like columnstore V1 in 2012. So, um, that, that, that’s, that’s quite disappointing there, but, um, you know, like, like, uh, the, the, the preview feature thing that there’s that database scope configuration for preview features. And it terrifies me because of what we were just talking about with like the timeline of things becoming GA, right?

So let’s say, uh, there’s a, there’s a preview feature that you want to try and use, right? And so you turn on preview features and then, uh, that feature, like another cumulative update gets released and new preview features. Get added, but your feature is not out of preview yet.

Like you can’t turn that off. Cause then you lose access to the thing that you’re previewing, but maybe like you, like something new in the preview screws things up for you. And so like, I’m, I’m terrified of this flag.

Like, I don’t know how to cope with it because I don’t know how Microsoft is going to treat things in it. Like what if new things get previewed that break for that break things for you, but you need access to the old things that you are previewing that were working for you. Like it all, it all scares me.

I’m trying to think of situations where I would be willing to use a preview feature in production. Yeah. And I would have to be real desperate.

Would it be query store on readable secondaries? Absolutely not. I’m just, I would have to be real desperate. Like I would need to, I mean, cause the thing with pre yeah. The thing with preview features, like you said is, you know, here today gone tomorrow.

Sometimes there’s like, there’s not a, it doesn’t feel like a safe relationship. No, no, it doesn’t at all. It it’s like, I’m, I’m, I’m glad that there is a way for people to access them.

I feel like we, we had that with trace flags, but who can remember trace flags, right? It’s like, uh, like when do I turn them on? Is it global?

Is it sessions startup trace flag? Like, uh, there’s a lot of, a lot of things that get lost in the brain, but the preview features flag, this is like, you know, turn that on and you get access to all the preview features. Yeah.

This is suitable for a development database. Correct. Yeah. Correct. But it still scares me. The other thing I think, uh, that I am excited about with SQL Server 2025, uh, standard edition, developer edition, standard develop developer standard edition, whatever it’s called the ability. If you are running standard edition in like a production environment and you want to know how it behaves without paying standard edition money for it, you now can actually run that on development instances.

Yeah. Uh, with a, an edition that, which, which I think is going to be like, just really useful for like, say you are like, what is my workload going to be like without the extra read ahead reads? Like, you know, there’s all of these little features in enterprise edition that add boosts.

Yeah. But you don’t really know what they are. And you actually want, like, let’s say you actually have benchmarking load tests. You can actually run those against developer standard edition. Or, or even, you know, even just for like a consultant like myself to keep straight, which things are available across which editions it’s, it’s nice to have.

Um, you know, like certain, like, like the Microsoft documentation is very strange about certain things. Like, um, I remember it, like, it took a lot of arm wrestling for anyone to tell me if optimized for sequential key was available in standard edition. Like it wasn’t documented anywhere.

And they were very hesitant to document that for some reason. And it was just like, cause like, like, like just cause like of the, of the way that it’s a feature of the engine, they couldn’t like, they didn’t want to say if it was standard or enterprise, like enterprise only or something. And I was just like, but I just want to know if it works here.

Like, like, like if I’m, if I’m evaluating which one is right for me and I know that like, maybe I, you know, I just want this one thing. Like, do I need to pay for enterprise edition to get this thing? That’s going to be important to me.

Or like, uh, if I’m trying to like consolidate and save money or like, like whatever, like whatever the situation is, like, what am I losing by going here? Like what things are now off the table for me? And so I think it’s going to be very useful for, from a number of perspectives, like, just like keep straight, like what, what’s available where and when, because the documentation doesn’t always tell you that in a straightforward way.

Yeah. Yeah, no, absolutely. The, the documentation is pretty darn good, but, uh, there’s a lot to cover.

Yeah. Yeah. Ever, ever more to cover. Ever more and an ever more product with ever more similar, but different, but similar, but almost the same name.

Yeah. They got, they got rid of an edition of SQL Server too. Didn’t they?

Like they got rid of web edition or something. Oh, did they? Yeah. Yeah. I think so. Like they got rid of one of them express apparently got some boosts. Like you get like, have like a 50 gigabyte database on that. Now no longer a 10 gig limit, which I, I continue to think about my fleet of production.

Express instances. Yeah. Always. With highly charted data on them that. There’s, there’s a very famous question on DBA.StackExchange, uh, where a gentleman, uh, wanted to have an express edition with like 10,000 databases.

And he was running into problems because the date it was taking a very long time to start up. And there were a lot of very strange solutions added in his, his potential fixes for that. Like starting up SQL Server with no databases attached and then attaching them in groups.

It like, I don’t think, I don’t even think 5,000 is a good number for that. I think there’s a lot of bad. 5,000 express editions with two databases each.

Hey, look, with AI scripting, all of your stupid ideas is now possible. Will it work? Yeah.

No. The sky is the limit. Is it going to work anyway? Absolutely not. No, I’d say, I don’t know, you know, maybe the sky isn’t the limit. Maybe like, maybe the ground is still the limit. Oh, like the real limit is always zero.

Well, so I thought that was, I thought that was a fun one, but I don’t know. I have, I have strange ideas about what’s fun these days. Yeah, me too.

I’m really into static websites. I’m really just into text files more and more. I’m just really into text files. Wave of the future as far as I’m concerned. Um, I, I, you know, I, and all your, all your website tanks.

Tinkering makes me want to tinker with my website because like, I don’t know. I, I, I’ve, I’ve like never been happy with my website. Like I’ve been like, like, like over the years I’ve made changes where I was like, okay, this is an improvement. But like, as a whole, I’ve always been like, man, I don’t want this.

I want, I want something else. Like, like why is, why is it all so hard? Like, why is it, why does this look that way? Why come when I move this thing, 50 other things moved in the wrong direction? And like, why is this a new block now?

Like, like WordPress just very, I don’t know, has always messed me up terribly. So like, I’m, I think, I think maybe, maybe I’ll use AI to like figure out if I could migrate my website from WordPress to, to something else. Maybe I’ll use Kendra.ai.

Maybe we can show you the way. I was, I was talking to Jeremiah the other day of my, my husband for folks who have not listened to past episodes. And, uh, I was describing like a workflow that I’m developing where like to do a newsletter, I can create a text file with the newsletter stuff that I want in it. And then I have a Python script that’ll talk to the, the API of the newsletter API thing and automatically set it up, use the template, do all this stuff, verify, and I can preview and everything.

And he was like, he said, you’re, you’re becoming a programmer. I was like, no, I just love text files. Yeah.

I just want everything to work off a text file. And I, and I love me a pull request. I love me a workflow where I can preview things in a branch, make sure it works before I merge into main. So I like text files and I like source control and I like websites where I can work with them in that way.

And I like school code where I can work with it in that way too. Yeah. I mean, there’s, there’s, there’s a, there’s a lot of upside to things working off a text file.

As long as you don’t have to like generate or maintain your own JSON, you think you’re, you’re good. Cause that’s really where things fall apart for me. Yeah.

I, uh, I don’t want to talk about YAML though. No, no, we don’t do that. We don’t do that here. We started, we started, we started getting into the ugly parts of life. This is a family friendly place, Kendra.

We don’t talk about YAML here. Yeah, we don’t. Um, well thank you for joining me to chat today about, uh, things I, who would have predicted that I would be like, let’s do a podcast. I want to talk about how much I love resource governor and standard edition and SQL Server, but.

And text files. And text files. That’s where life has brought me. And I’m glad, I’m glad that you feel similarly. You are dangerously close to just writing your own database at this point.

Oh man. So like early, early in my career, there was. It’ll be a zip file. Oh my gosh. There was a development team I was working with. And one of the developers was just like, you know, we should just write our own storage engine.

And at that point I just like, I was like, I’ve got to go now. And I was like, we’re done. No.

Yeah, no, I’ve, I’ve, I’ve, I’ve talked to people who, who also think that’s a good idea. And you’re just, you know, it’s always like, you know what? Like, I have no doubt that you could do that and it would probably be okay, but I don’t want any part of this. I, I have already witnessed how much work it is for a lot of really smart people to maintain.

A storage engine that other really small, smart people wrote. I, it’s not, that’s not a small pet to take on. No, no, no.

That’s like, like, like even just like, like not, not even forget like your own storage engine. Like you ever seen those like, like, like code exercises that are like, write your own file system. And you’re like, no, no, I, I don’t want to.

I like permissions blocked file. No, no, I’m not doing that. No, no, not today. Nope.

Not any day. Not this lifetime. Maybe another one. Perhaps that’s something that like, you would have to be one of those, like, like seven year olds. Who’s really good at coding to get into that sort of thing. But I don’t, if we want to do this, we need to find a seven year old.

The only person with enough optimism, hope for the future and naivete to take this project on and be successful. And also enough, enough years ahead of them. They’re going to need all of that lifespan.

About 80 years. You’ll be good. All right. Well, where do people find you on the internet, Eric? Oh, you can find, what is my, what is the website that I hate its name? EricDarling.com.

I’ve got, I’ve got various social medias like YouTube and LinkedIn and Twitter and blue sky. So you can, you can find me pretty much anywhere that you show up and drink and say embarrassing things. And you can find me on Kendra little.com because I finally obtained the URL that is my first name.

And then my last name, I no longer have to be last name, first name. I more little Kendra. Kendra little.com.

I have achieved all of my dreams. Congratulations. Thank you. I’m pretty excited. So y’all have a good one and see you on another episode of Dear SQL DBA again. Goodbye.

Going Further


If this is the kind of SQL Server stuff you love learning about, you’ll love my training. I’m offering a 25% discount to my blog readers if you click from here. I’m also available for consulting if you just don’t have time for that, and need to solve database performance problems quickly. You can also get a quick, low cost health check with no phone time required.

bit Obscene: SQL Server 2025 Developer Features

bit Obscene: SQL Server 2025 Developer Features


Video Summary

In this video, I delve into some of the new developer features in SQL Server 2025, discussing both the practical implications and potential pitfalls. We explore exciting additions like regular expression functions and JSON handling, which promise to streamline development but also come with their own set of considerations. The conversation touches on how these features might be misused—like invoking external REST endpoints in triggers or using them excessively in stored procedures—and the importance of understanding when such practices are advisable versus when they could lead to performance issues or other complications. We also touch upon Microsoft’s approach to documentation and guidance, highlighting that while they provide technical details, more emphasis is needed on practical advice to prevent misuse by developers.

Full Transcript

All right. So it turns out Sean is not going to be late for his interview to Valet Park Cars at the Olive Garden. So he’s able to join us for at least one more episode of the Bit Obscene radio podcast. In order to make Sean feel like a complete outcast, Joe and I have decided to formalize our attire as well. In this episode, oh, we have an and. We’re going to interrupt the intro, because Joe has something important to say. All right. I do. Sean’s man’s term it took so long to dress them. I was able to learn how to tie a tie and tie it before Sean got back. Yes. I wish I had that recorded. Joe nearly died twice. Both of you look amazing. Thank you. This is this is a lovely, I think, Paisley. Sort of. Yeah. Paisley tie. Paisley. All right. Cool. Yeah. Well, it’s a pattern. Okay. All right. Okay. Anyway, this continuing on the theme of SQL Server 2025 week, we’re going to talk about new developer features in SQL Server 2025. And apparently Joe is finally going to pick up the slack on something that is not a hangman’s knot around of a tie around his neck. And he’s going to talk about some of his favorite new features in SQL Server 2025. So take it away, Joe, now that you’re now you have now that you have now that you have to be a know one to interrupt. Hopefully you can think of something to say. I don’t know about favorite. You know, I was trying to find some new stuff to talk about. You know, it’s it’s a change event streaming didn’t just jump out. Like, you know, dig at the bottom of the barrel for some of this stuff. Yeah. No offense. I mean, I don’t take it. I hear that SQL Server 2025 reinvents the database developer. That’s that’s what I hear. Wow. It has been said. Wow. It has been said. But we’re I’m not gonna. I’m not I’m not ready to be reinvented. I feel like learning how to tie the ties enough reinventing for a while for me personally. So I’m gonna talk about the the more mundane things, you know, the kinds of things that the the ivory tower people with fake jobs used to these school that’s for database developers about where like we want to do like, regex and the database and they’d be like, ah, that’s using too much CPU SQL Server licensing costs. What kind of dumb ass would ever want to do regex where your data is? Obviously, you should send a billion rows to the clients and do the do the regex there and think about savings.

savings. Or I mean, like if you really think about it, like we only have one SQL Server. Well, that’s the way I think about things. One SQL Server at a time. All of our end users, they have phones and computers like think of all those untapped resources. You know, instead of doing the processing ourselves, we should just make people’s phones overheat. It’s almost like a cloud or Silicon Valley episode. Yeah. Um, with all that said, um, even if it’s perhaps not the best practice, it’s important to go where the people are. If you have, you know, 10,000 poor developers, all poorly re-implementing regex or fuzzy string matching in the database or CLR, if Microsoft can provide a built-in, which is better than, uh, hash flights, then I’m all for it.

And I mean, regex, right? It’s like, we have a few of those things in SQL Server 2025 regex, fuzzy string matching, something I’ve never used, but I’ve had to tell myself that that doesn’t sound like a lot of fun. Um, seemed like a rest endpoint kind of got to me.

Well, it’s funny that you say that because. Is it? Yeah, well, it is funny. Okay.

It’s funny because I’m reading the little bullet point right now for that. And it says you can have data processed via an Azure function using that, which I don’t, which I don’t doubt is true, but you can do the same thing for change tracking, which seems way more natural of a method. And only if you have change tracking enabled.

Well, you just have to enable it for, for, for the table you, I care about. I mean, like if the difference is I have a Q table, I enable change tracking, I inserted it in my Q table, and then I can call an Azure function on that row. Like that’s great.

I mean, compared to like create some goofy file or something, or. Did you, did you know if you enable change tracking and you use an availability group, your database has to restart an extra time? Yeah.

I did not know that is, but can’t you like not restart in the cloud? So it’s not. Nope. A problem. No, it’s just a different problem. Well, it sounds like Sean is advocating for SP invoke external rest endpoint, which is personally very surprising to me, but it’s nice to be surprised sometimes.

There’s also some stuff about JSON. I’m not really a JSON guy who has a JavaScript name, you know, like keep JSON where the JavaScript is inside of SQL Server. But I suppose I suppose I’m being one of those people who is, you know, the tut tutting that you’re not.

You’re not. You’re very English. You’re polishing the ivory on that. Yeah.

I have a question for you, Joe. I’m going to withdraw that one. Go ahead, Sean. Hit me. Yeah. You might be a little too young for this, but do you feel like, do you feel as though 2025 is 2005 all over again? I didn’t really work in 2005.

Okay. When I started the SQL Server, I believe we had to support SQL Server 2005 because we had a few customers are still on it. So we couldn’t use some of the wonderful 2008 features.

Like row numbers. Well, I was going to say, let me know. Like some of the, maybe all the window functions. Does anyone, my question is, do you remember SOAP endpoints?

Yeah. I remember SOAP endpoints. I do not. I was going to say, Eric, go ahead. Yeah. No, I don’t want to. Exactly.

I want to not SOAP endpoints anything. That’s, I’m good on that. We can, we can cut that right there. It feels very SOAP endpointy. I mean, Joe, I know you, I didn’t want to steal your thunder, but it, there are, you know, you were saying, you don’t want to be an advertiser telling people, you know, tut tut, and then you didn’t want to tell them.

I tut it. But at the same time, I think we all, at least on this call, understand that there are things to use and there are times to use them and narrow, very narrow times to use some of them. And then there’s what people do.

Mm-hmm. Yeah. The, the, the message about the narrowness doesn’t tend to make it too far past the screen. Right. It, it tends to get lost in transmission in a lot of places. And you’re like, you know, like I I’ve seen like, so like the SOAP endpoint thing never really hit me too hard because I didn’t really run into a lot of people doing stuff with that.

But what I got hit with on a bunch of stuff was the places that people would cram XP command shell calling external executable services, things like that. And like, I just feel like the external rest endpoint is just like, it’s okay to do that. It’s like it rubber stamps the whole brand of stuff that just people are going to do real bad.

Yeah. Yeah. Yeah. I’m looking at the announcement that says call rest graph QL endpoints.

Mm-hmm. Have data processing Azure function. Mm-hmm. Update a power BI dashboard. Apparently.

That’s an interesting one. Uh, call an on-premises rest endpoints. So do literally whatever you want. Yep. And you can talk to Azure open AI services. Those are, they’re like best, you know, at the front new use cases for this. Yeah.

Yeah. Yeah. It’s like, how else do you talk to your girlfriend? It’s impossible. The problem I have with the, uh, tut tutting is people who just like ended there, they don’t give like practical solutions and by practical solution, I don’t mean why don’t you just rewrite the whole application that someone else developed 20 years ago.

Like, I, I, I, I am all for telling developers, here’s a way to do something. Here’s some considerations. You’re supposed to be smart and well paid, figure out what the right solution is as opposed to just the, you know, well paid.

Yeah. Yeah. I wouldn’t want to send in a, a, a large comma separated list and split it in XML in my story procedure.

It’s not database appropriate. You need to rethink your entire design because who would ever want to do that? I mean, you know, you know, I mean, you, you are, you are just channeling Joe Silco right now and we all, we’re all here for it.

But, you know, I, I, I do agree that it is, it is, it is in impolitic to, uh, castigate people for doing certain things without offering secondary solutions. You know, it’s, you, you, you catch that in the SQL Server community quite a bit, especially with things like no lock hints.

Everyone loves to gang up on no lock hints, but no one really offers you a secondary solution for all of the terrible blocking that goes on under the default read committed garbage isolation level.

So I, I understand, I understand your position on this. And I, I think, uh, the, the, the issue for me with things like, uh, external endpoint invocation is it’s going like, and like, it’s funny cause like the regular expressions functions thing is like right above it.

Uh, and JSON is right under it. And so for, for me, what’s, what’s funny about it is that like the documentation is never going to tell you how not to use it. Uh, and this goes back just be like beyond like crazy, like features like this, this goes back to like, you know, other functions where it’s just like, like is null and coalesce and stuff where no one, nowhere in the documentation.

Is it like, Hey, this is a nice presentation there function for you to get rid of nulls with, uh, don’t maybe use this in your joints and where clauses. Cause it’s not going to go well.

Right. So you have like, like you, like all, all the, all the documentation does is like give you some of the technical details. It doesn’t say you probably shouldn’t do this with it. And, and I really wish that like, like, like a lot of blog blood is going to have to be spilled and telling people what they probably shouldn’t be doing with this and like where it’s going to hurt before any sort of message starts getting across.

So that’s, that’s my thing about it. I mean, that’s a real concern. And I, I know you all know I used to work in support. I have worked in support for quite a long time, but I used to work in support and I would, you know, try to help people with, with different things.

And, uh, one of the, I, I, this is an actual real thing. One of the items, uh, I got asked to do from an actual per, uh, you know, customer on SQL server was to document that you should not shut off your server in the middle of it running.

And that you should like, you should not pull the power plug to it. And that if you wanted it to talk to other servers on the network that we needed to document, that it actually have a network connection because none of the documentation says that it won’t.

Right. And so my only, my only thing there is there’s so many things, you know, you, you, you tell people, okay, Hey, you know, it’s, it’s a medical commercial. If you’re allergic to blank Brzenica, don’t take blank Brzenica.

Okay. Well, how do you know if you’re allergic to it? Yeah. You, you gotta, you gotta, we gotta say like, Hey, don’t divide by zero. Hey, don’t try to run invalid instructions.

Hey, don’t try it. Like, you know, there’s some things that are just, you gotta, you gotta take it to find out if you’re allergic to it, you know, but there are just some things that I think you should understand.

It’s a very bad idea. Yeah. Right. Like if, if someone says, Oh, a blocking function that I can call, this sounds great. Let’s call it in a tight loop for a billion items. Or let’s, let’s invoke an external rest endpoint in a trigger and see what happens.

Like every time we change data in the table, let’s go update our power BI dashboard. Yeah. Question for Joe.

How many triggers have you seen where someone has done something like that? Hmm. I’ve seen a trigger on a view that called a CLR, but that was a, which, which, which, then called in the Azure blob storage container.

But that is a temporary solution that we’re doing as part of a migration. Oh, so by, by I’ve seen it in a trigger, that means you wrote it. I didn’t write it.

Somebody wrote it. Don’t know who. Wasn’t me. I mean, if I had been asked to write it, I, I would have written it because it’s one of those, you know, this is a bad thing we’re doing. We’re doing it temporarily.

It’s, it’s gonna get us to a better place. So be it. Um, and, uh, to be fair, the old data is in. I actually have the name of this. It’s like the, it’s the future we use that like no one else uses the.

Stretch tables, uh, SQL Server data files in Azure blob storage. Oh, you. Oh, wow. Yeah.

So IO performance is very terrible. Yeah. So like going from that to, oh, well, we’re gonna. Call in the blob storage already using blobs. So whatever, uh, to Sean’s point. It does go the opposite way sometimes in that.

Yeah. Former job. We were complaining to Microsoft that he had, I think it was. He had two or more. Defined resource governor pools.

On big servers, you could see some really awful behavior. There was some kind of like memory thing. And I, I guess we asked so much about it. And we’re so annoying at that.

That, that someone said they were going to update all the documentation to say that it wasn’t supported to have more than one defined resource governor pool, which is a really interesting approach to take. Cause, uh, you know, like, I kind of feel like that should have been apparent from, from the beginning, but you know, it ended up not happening. Uh, cooler minds prevailed.

To what Eric said. I need to make sure I, uh, share the right screen here. It’s gonna be a real disaster. I could. Um, yeah, there we go.

Uh, age filter popping up. Podcast over. Yeah. All right. Good. I, I think when we, we, we won’t have to add that out. Um, broad impact potential. I feel like this is one of the, like a good examples where Microsoft actually is saying, Hey, maybe you shouldn’t use this in certain ways.

And here are some things to what, like career-term hints are best used for short-term fixes. So I feel like in career- Nothing is longer term than a short-term fix.

Like your, your, your thing in a trigger is forever now. Change this before 2012. Yep. Um, yeah. So speaking of a short-term fix, I’m, I’m blurry again.

You are. So just wave your, wave your hand in front of the camera. Like you just don’t care. Yeah.

Ah, there we go. Yeah. So, sorry, I got, I got you off topic though. What do you, what do you feel or how do you feel about Jason being another, now you have, you have XML, now you have Jason. Uh, people are already doing it.

If they’re already doing it and they’re not going to stop, you know, as we’ll make it a little bit easier. Yeah. Have you tried any of the functions with it? I’m, I’m not a Jason guy. Uh, I messed with it a little bit because I was curious if, um, uh, Jason, uh, like, I was curious if like the Jason indexes would, um, be responsive to page compression and they are, which I think is probably a good thing given how large the, the Jason stuff is.

Cause I mean, look, like, like, you know, like Joe said, um, people are already doing it and using it. And it’s, uh, it’s a real pain, uh, especially because people tend to use all this stuff relationally at some point. It’s like, you know, um, my, my, my feelings on Jason and XML are largely that, uh, it’s so like, it’s fine if you want to store that in the database, but then if you’re doing anything else with it in the database, you should probably reconsider.

Like make maybe occasionally just have to fetch it out. But, um, you know, I, I’ve, I’ve dealt with all sorts of applications where it’s like, we’re going to store this XML in a column. And now every time someone logs in, we’re going to parse this XML for like the user ID element and join on that.

And the server is going to crash whenever more than 20 people log in at one time. So like I’ve dealt with a lot of bad stuff as a result of being able to store these, uh, these blobby things in the database. So, you know, um, like really anything that can be done to make it less painful in the database is great, but, you know, um, developers being developers, uh, how well they keep up with these things and how well they use these things is yet to be seen.

Um, you know, I, you know, like, like, like the big thing with like, I mean, it was true with that. It was XML. It’s going to, but it’s more true with JSON because JSON is more popular with developers.

I think, uh, at least seemingly so, uh, is that the reason they’re in love with it is because there is no hard and fast schema rules for it. And so like, you know, the, the, the typical, you know, consult in the, you know, polish the ivory thing is like, like, it’s cool if you want to store the JSON there, but querying it in this way is going to suck. Like, if you want to do this, you should really parse it out to a relational table where you can, you know, have all the data separated out.

But developers love the sort of non-conforming schema aspect to it. And so they always want to like add stuff to it or maybe even occasionally take stuff out of it. So like even something like that isn’t always practical.

So, um, we’ll, we’ll see how well this stuff ends up getting used and we’ll see how well, um, developers end up treating, uh, JSON in the database. But, um, you know, my, my, my typical view is not too sunshiny on these things, especially given the, uh, very long lead times on people getting to new versions of SQL Server. You know, I think that goes to Joe’s point, right.

About the, you know, when he was talking about the regex, right, Joe, like it’s, you know, people are already using it. They’re already doing it. Yep. It might not be the best use. We, we might have to, or should there should maybe be some warning signs and something like that.

But if you’re already doing it, why not? As Joe was saying, maybe there’s less crap having 50 people do 80 implementations of it because they copied and pasted something off a stack overflow 16 times. And that’s what’s production code now.

Well, I mean, now, now it’s whatever the AI spits back is production code. So stack overflow is far, but, but I think, um, Joe, I had a question for you, you know, with the regular expression and fuzzy string matching. Do you think that that is going to offset a lot of the full text related queries?

Do you think that’ll hurt full text usage or full text usage was already pretty bad to begin with? It’s hard to hurt full text usage. It’s a tough, that’s a tough one.

My understanding of how full text is supposed to be used is you, you like, take a static database backup, you have a ton of data, and you build your full text indexes on that. Then you have your like researchers or whatever query that data, because it’s not, it’s not synchronous. So if you’re ever looking at some like, oh, I’m going to just build in a full text type of string search in my real time application.

As far as I know, there’s no way to make that synchronous. So you could get what could be perceived as wrong results. Like maybe that’s wrong.

I’m ill informed. It’s an asynchronous process. It has incremental. Yeah. Right. It does. But yeah. I think it could be great if, like I said, you have a static database. It’s not changing.

But other than that, it seems like that’s the only real use case. So I don’t have a lot of experience with it. It’s probably misused a lot.

So to your question, if this lets people misuse full text indexes less often, then great. I mean, it sounds like it’s going to be a little bit better, right? The fuzzy string matching.

I think we all have a friend, you know, who, who spells the word color incorrectly and color, behavior, labor, all sorts of things. Exactly. So I think there’s a lot of small areas where it’ll chip away, especially because full text really hasn’t had an upgrade in a while.

The. Well, what would be the point? Yeah.

Like who would, who would, who would benefit? Right. Make it a synchronous. Then finally we can, people like me can stop hating on it. I mean, but I think the performance around it is awful. It’s like, it’s good there.

But I think the, you know, the regular expression stuff, I’m not a personal fan of people using regular expression in the database. And mostly because when you start using regular expressions, you go from having one issue to a million issues, right? Especially the way that it works.

It’s not magic. I mean, I guess you can call it magic, but it’s not magic. And there’s a good. As far as I know, the implementation is done with one of the open source. I think Google’s open source regex model, but it’s, you know, it’s still requires memory, CPU, all those things.

Just like Eric was saying with the XML, like you might, you still may have to use DOM and the other stuff. Especially like the JSON stuff. Do you know off the top of your head?

Because just because I haven’t looked at it yet, but I know that like try parse and format are basically like, like CLR thin clients in a weird way. We’re like, like, like they, like they use CLR very heavily in their implementation. I don’t know if regex does this.

Like I have to look at regex is not CLR. No. Okay. I was going to say like, wow, that’s, that’s surprising. Yeah. That’s, that’s, that’s quite surprising, but yeah, cause I was, I was expecting to see like, like a query use, like with like, you know, regex is like, you know, used heavily in it or like using like, like, like a big where clause or something just to like rack up CLR time the way format does when you use it there. So, no, it should actually be pretty quick.

It should, it’s going to be native. It should be very quick. In fact, it should be faster than the CLR implementations that people have done. No. I mean, Hey, if, if, if you have the guy wearing a suit, tell you that, then it’s going to be a man, looking serious in a suit. It’s going to have regex all over the place.

Yeah. You know, and I think, I think the things that regex opens up are, are useful to developers generally. Um, you know, there are some check constraints that no amount of like car index, pad index, sub stringing you do can never, you know, get you what you want. Uh, and I do think that, you know, like, and it’s one of the, it’s one of those things where like clearly Microsoft is, has to make up messed up, has to like kiss and make up with a lot of customers in Azure who can’t use CLR for stuff.

Like, you know, like you can’t like Azure SQL database is like no CLR anywhere. So I, like, I do, like, I do realize that some of this stuff is kind of kiss and make up territory. Like we, like they have to build it into the product cause people want it.

So like want it so badly. But, um, I think, I think it is nice to have, I think what, um, people are going to do with it is going to hopefully, uh, keep me employed for a few more years. So, you know, Joe’s gonna, Joe’s gonna wave his finger at me again, because it’s, it’s, you know, one of the things you just said, there was a check and strength that no amount of, and I just thought to myself, why are you waiting till the data gets to the database to check it?

Joe Piazza- Can, uh, can’t, can’t trust the client, man. You know, untrustworthy those clients are these days. Joe Piazza- Because think about, think about people who bypass clients.

Um, like one of the, like, honestly, like one of the big, the big things that I see is you have like, and I mean, this is a somewhat related, but not quite related story was, um, one of my clients, uh, we were reviewing trigger code and they had sort of like the classic, like this trigger is designed to only handle one row at a time thing. Joe Piazza- Uh, and that was fine because the application guaranteed that only one row at a time would ever have a thing done to it. Joe Piazza- But they had all sorts of like admin users who would do whatever they wanted.

Joe Piazza- And so like they had lots and lots of rows in the table that were all messed up because the trigger only fired for the first row that they did something with. Joe Piazza- So like they would update like a 500 rows or something and only one of those rows would have a trigger result, like, you know, done with it. Joe Piazza- So like I, I, you know, there are things where you do have to protect the data where it lives.

Joe Piazza- And regex is potentially one of those things that can help keep lousy rotten data out of your database. Joe Piazza- That’s a way better example than the one I have. Joe Piazza- I was going to say some junior dev copying and pasting a generated code.

Joe Piazza- And, uh, do you think the AI is considering your, uh, your, uh, your data integrity? Joe Piazza- No, but I mean, it’s definitely not. But the, I guess the thing with that is why are you letting admins use like management studio to update stuff? Joe Piazza- It’s not me letting them like, Eric, why are you letting them look like, look, man, there, every company is going to have this stuff going on.

Joe Piazza- Like, like, like, look, we have big time, man. Joe Piazza- Look, uh, as much as I wish that everyone had a reasonable set of store procedures that everyone was required to use in order to interact with the database, that is just clearly not the situation at a lot of places. Joe Piazza- There are not that tight controls on things.

Joe Piazza- And, you know, like, I’m not saying it’s the wild west and at every, every single one of these places, but there’s a certain amount of like, well, this is a special situation. Joe Piazza- We’re gonna, you know, I don’t know, like, whatever. So, but these things happen like over the course of, I get it. Joe Piazza- I’m just playing devil’s advocate here of, you know, it’s the, we go from, as Joe called me out, we go from one end of like, you know, don’t, don’t do this and stuff to the other end of well, there’s all these special cases, but then to, well, we should know about all these special cases in the optimizer to well, maybe we shouldn’t, you know, everyone has the thing. So I get it. I’m just.

Joe Piazza- It’s not the same thing at all, man. You can make the optimizer better for everyone versus, it’s not the same thing at all. Joe Piazza- One thing that you guys could do is when you’re like creating a login or whatever, maybe have like a don’t allow this login to connect to the SMS, like built in thing. Joe Piazza- Just an option.

Joe Piazza- Yeah, no, I mean, really like something like that. Joe Piazza- That’s what logon triggers are for. Joe Piazza- Why do you have a login? Joe Piazza- Yeah, you can do it yourself when Microsoft can do it better. Joe Piazza- That’s true.

Joe Piazza- That’s true. Joe Piazza- Yeah. Joe Piazza- Because yeah, like in a lot of cases, you have these code deployment tools, they ask for high privileges. Joe Piazza- Mm-hmm. Joe Piazza- And now your deployment people have a high privilege login they can use, like unless you have login triggers like Eric said, or similar solution, like how are you going to keep people out? Joe Piazza- Yeah.

Joe Piazza- I mean, there’s actually, no, there’s actually a whole, a whole big topic. Joe Piazza- I don’t want to segue because we, that’s a whole big topic we talk about. Joe Piazza- Oh, that’s okay.

Joe Piazza- You know, we’re talking about developer features here. And we didn’t talk about any type of seamless access to secrets or other credential stores outside of, you know, EKM or, or AKV specific stuff. Joe Piazza- With the external rest endpoint invocation. Joe Piazza- That might be a valid use.

Joe Piazza- You might be the only person who knows what that means. Joe Piazza- I need you to consider that. Joe Piazza- The way I thought you were going was, you could like export data through that and that’s bad, but you seem to be saying the opposite. Joe Piazza- Well, no, what I’m saying is, you have, you’re saying, you know, now you have these high privilege users, right? And you don’t want to have these high privilege users.

Joe Piazza- And I think most people are pretty familiar with store procedures, your module signing, counter signing, stuff like that, you know, maybe not. Joe Piazza- I know one of the things that comes up a lot is, can I log, can I log in with a certificate, right? Joe Piazza- Can I just give this certificate and log in? And that’s not, I wish that were in there. It’s not in there.

Joe Piazza- It would have been great if it was in 2025, maybe, you know, hope for 2020 V next. Joe Piazza- But I think that’s one, one way to help with that, especially if you have systems that are pretty well bogged down, you can export certificates, etc, etc. Joe Piazza- Or if that certificate isn’t stored in some type of hardware storage module, right? AKA like an e-cam or something.

Joe Piazza- But, or that’d be an HSM, e-cams, the framework to get to the HSM. But the, you know, the REST API endpoint, I mean, could you not call, Joe Piazza- Something to get a credential stored somewhere and you need to be able, you know, you have to impersonate a different user or have a special login or do whatever. Yes. Joe Piazza- You can do that. And then is to your point, Joe, of like the data exfiltration. And this is definitely not an enable this and use it everywhere. But the, you know, the changes in data encryption, right? So you can have the Joe Piazza- Oh my gosh, I just had a stroke. Hold on.

Joe Piazza- Happens to the best of us. Joe Piazza- Yeah, there we go. All this stuff is too complicated for me. I like how they have a data ad and you can use big int. Joe Piazza- Yeah.

Joe Piazza- Like that’s the kind of meat and potatoes feature that makes me happy or a substring. Joe Piazza- Yeah. Joe Piazza- Where you don’t have to write some goofy code to figure out the length because you want like the whole string except you’re chopping off the front of it. Joe Piazza- Yeah.

Joe Piazza- All right, Sean, how is that? They’re still for enough time? Sean O’ The product function so you don’t have to remember all that log math. Joe Piazza- Yeah, yeah, right. The product function. That’s a good one. Joe Piazza- Yeah.

Joe Piazza- So one thing that was lost on me, and I’m sure we have viewers who are even younger than me, if you can believe that, is you seem to be alluding and referencing. Joe Piazza- Some soap endpoint crisis? Is it something that just got discontinued or like what was the big bad thing that happened with soap? Joe Piazza- It was blockchain all over again.

Joe Piazza- Oh, it just ended up being overhyped and useless and just taken out? Joe Piazza- So soap was the simple object access point or something like that. I forget, I’ll get roasted for it, it’s fine. Joe Piazza- Essentially, it’s a rest endpoint, but you get your object and some other information about it, and you can invoke it.

Joe Piazza- You can think of it as a rest endpoint, but it was another one of those things where that was the hot technology for the blockchain. Joe Piazza- It was the hot thing. It had to be in McDonald’s Happy Meals all the way to your toaster to every application everywhere. Instead of saying, how can we use AI? It was how can we use the blockchain? We need to be on the blockchain. So it was kind of that. It went to the wayside. There were other frameworks that ended up kind of coming in. And I also don’t think that this is, again, a very strong opinion that I hold. Joe Piazza- I don’t think you should be having a database server just with randomly punching holes for various endpoints that do whatever on them. And I know I say that as someone who loves HA, and we have a very specific endpoint for, but I don’t like, you know, now you have a, like, there’s nothing, for example, there’s nothing stopping anyone from creating another TCP endpoint for TDS. Right?

Joe Piazza- You can create 17 endpoints if you want. In fact, a lot of people don’t realize you can actually NUMA affinitize your endpoints. So yeah, I know exactly. I didn’t. It’s a very advanced setup. Joe Piazza- Yeah, I think a lot of people might never even think of that. Now you’ve ruined it. Joe Piazza- Yeah. I was rewatching Joe’s columnstore loading.

Joe Piazza- Oh, well, thank you. I appreciate that. Speaking of things people might not have thought of, I don’t know if you saw this, Sean, but about the endpoint thing. I had the brilliant slash horrible idea to, if you start up and extend it, you know, Joe Piazza- It’s really, it’s really hard to do. Joe Piazza- Yeah, I think that’s the kind of thing.

Joe Piazza- Yeah. Joe Piazza- Yeah. Joe Piazza- Yeah. Joe Piazza- And then, you know, I had the same thing, right? Joe Piazza- Yeah. Joe Piazza- Yeah. Joe Piazza- Yeah. Joe Piazza- So, like, it was already possible to just create, like, arbitrarily create files with the filenames you wanted in a blob container. Joe Piazza- And then, you can trigger based off of those files getting created using, like, Azure Functions or whatever else.

Joe Piazza- So I already had, like, a workaround to the, to the endpoint thing that you guys are officially supporting. Joe Piazza- And I was, I was gonna do it, too. Joe Piazza- I was gonna do it.

Joe Piazza- Believe me, I was- Joe Piazza- Just dastardly kids. Joe Piazza- Yeah, but then, now I don’t have to because, well, it’s funny because I, I got scolded by some guy and now you guys just make it easy to, you know, I was scolded is like not do it. Joe Piazza- I don’t know if the subject was like trying to get a like, like a PDF files or whatever on Azure SQL database.

Joe Piazza- What’s wrong with you, man? Joe Piazza- Yeah, right. Joe Piazza- Who’d ever want to do that?

Joe Piazza- And there are all these people, you know, the question got closed because who’d ever want to do that? Joe Piazza- And now you like look at the 2025 notes and it’s like, hey, you, you want to get some data, a SQL Server from anywhere you want? Joe Piazza- Yeah.

Joe Piazza- Or hey, you want to export some data? Joe Piazza- Oh, first one’s free. Joe Piazza- Anywhere you want? Joe Piazza- Yeah, with the first, first, first 100 megabytes are free, you can fit a lot of data. Joe Piazza- And export you long time.

Joe Piazza- 100 megabytes. Joe Piazza- Especially when you base 64 and code it with that. Joe Piazza- I’m sure that’s a really big deal for the people who need us. Joe Piazza- I’m not included in that group, but I’m sure.

Joe Piazza- Maybe Joe get rid of hash bytes, go to base 64 and code. Joe Piazza- Just make hash bytes scale. Joe Piazza- You don’t have to change anything. Joe Piazza- Just make it scale.

Joe Piazza- That’s it. Joe Piazza- I think you know, there was a time and place for that and it’s gone. Joe Piazza- Well, I’m the V positive and give you guys some credit. Joe Piazza- Because at time zone performance sucks on 2019.

Joe Piazza- But it’s like four times better in Azure SQL database. Joe Piazza- Just move to the cloud. Joe Piazza- Yeah.

Joe Piazza- That’s all it takes. Joe Piazza- Just sell your soul and move to the cloud. Joe Piazza- Well, you ended up putting on 2022 too. Joe Piazza- So for all you at time zone sufferers, there’s like the end of the title. Joe Piazza- I wanted to hit up just real quick.

Joe Piazza- So obviously there’s the new Chinese government coalition, which Eric had specified. Joe Piazza- But do you find, do you all find that I still in what I see and I want to get your thoughts. Joe Piazza- A lot of people are still mystified by coalitions.

Joe Piazza- Oh yeah. Joe Piazza- 100% I am often mystified by coalitions. Eric D’ I’ve never worked in a sophisticated enough environment where we changed the coalition. Joe Piazza- So I’m the wrong person to ask.

Joe Piazza- Like I know that there’s a lot written about them, but I find that my computer screen just like fuzzes over every time I try to read about them. Joe Piazza- The only thing that I have ever done to any effect with a coalition is use like Latin general bin to to speed up string stuff. Joe Piazza- Aside from that, I’m like, I’m like extended character special characters like like code page like sorting like I like, like I use whatever I just use the case sensitive version of whatever the default coalition for SQL Server is.

Joe Piazza- And I get hounded by certain New Zealanders who are like, that’s not a modern coalition. Joe Piazza- You should be using the SQL whatever one and I’m like, man, I like, like, okay, like, like, like, like, look, you’re right. Joe Piazza- But damn, I don’t know what I don’t know what else that’s gonna mess up for me.

Joe Piazza- I don’t know what it’s gonna fall apart in my life if I start using a new coalition. Joe Piazza- I have one client who has to use a Turkish coalition for one thing, because of where certain things come from that they sell. Joe Piazza- So that’s weird for me.

Joe Piazza- They love these names, huh? Joe Piazza- Yeah, so there’s a, I don’t know, there’s like weird swords in their alphabet and stuff. Joe Piazza- And so like, I don’t know. Joe Piazza- It’s all, it’s all strange. Joe Piazza- But yeah, I have no idea.

Joe Piazza- Like the UTF-8 stuff people people asked me about it. Joe Piazza- And I’m like, it’s kind of like, like Unicode and non Unicode had a baby and the baby had like seven fingers instead. Joe Piazza- It’s so bad.

Joe Piazza- It’s so weird. Joe Piazza- Yeah. Joe Piazza- Have you all heard of UCS2? Joe Piazza- No. Joe Piazza- So that’s what SQL Server typically uses for the encoding. Joe Piazza- Ah.

Joe Piazza- Which does support extended character sets, right? Joe Piazza- UTF-8 is interesting in that you have, if you look at the UTF spec, and I’m not an expert on the UTF spec, but if you look at it, there are all kinds of ways that you can have one character take up hundreds of bytes. Joe Piazza- UTF-8 is supposed to be essentially eight bytes, which is why it fits into a lot of places really nice.

Joe Piazza- Obviously it can be more, it can be less, but then you have UTF-16, UTF-32. Joe Piazza- Everything has one of those. Joe Piazza- Yeah.

Joe Piazza- But that’s what I was gonna ask, because like the binary correlations, I know a lot of people don’t like them for matching because they don’t get the case and sensitivity that most people typically want. Joe Piazza- But then they’ll say something like, well, it doesn’t sort right. Joe Piazza- I want to sort this one specific way.

Joe Piazza- Yeah. Joe Piazza- But I mean, look, they, SQL Server 2025 doesn’t add or fix any of those problems. Joe Piazza- So it’s like, you know, it’s just- Joe Piazza- No, but I was wondering about the correlations given the new one.

Joe Piazza- Oh, yeah. Joe Piazza- Oh, yeah. Joe Piazza- Well, I mean, look, I’m not gonna use a Chinese correlation, so I don’t know. Joe Piazza- This is America, Sean. Joe Piazza- Yeah, I do not have a use for a Chinese correlation at this point in my life. Joe Piazza- So I have no idea.

Joe Piazza- You know, I’m like, and I’m, when I look at like the new developer stuff in 2025, it’s like, like the double pipe string concatenation. Joe Piazza- I’m like, I’m sure that’s great for someone like, like somewhere out there, like some someone who loves Postgres is like I want like the double colon cast to a thing somewhere. Joe Piazza- And like, I don’t know, current date, fine. Okay, like, I’m sure that’s gonna really help people migrate from MySQL to SQL Server.

Joe Piazza- Like some of this stuff I don’t, I don’t fully understand like who the target audience is for some of these, like maybe like ORM developers or something, but it’s like a lot of it is sort of mystifying to me. Joe Piazza- And then, you know, like all like the distance and similarity functions and like, like, I know there’s someone out there who is like, finally, at long last, I have to be able to do that. Joe Piazza- I don’t know who the target audience is for it for that. Like, like sound decks and difference got us a very long way. Like why? Yeah, why are we here?

Joe Piazza- I think the fuzzy string definitely goes on top of that. I mean, preview features aside. Joe Piazza- Yep. But we’re going to talk about preview features next. I think we’ve we’ve killed SQL Server developer features. Joe Piazza- We ran, we ran through the entire list and we’re left wanting. So let’s let’s let’s nix this one here. So Joe can get his gas station pizza before it’s 10 o’clock at night. And we’ll we’ll we’ll do a potpourri episode for it for a short potpourri episode, so that everyone can yak about their their favorite and least favorite things. All right. Goodbye.

Joe Piazza- Bye. Joe Piazza-

Going Further


If this is the kind of SQL Server stuff you love learning about, you’ll love my training. I’m offering a 25% discount to my blog readers if you click from here. I’m also available for consulting if you just don’t have time for that, and need to solve database performance problems quickly. You can also get a quick, low cost health check with no phone time required.

bit Obscene: SQL Server 2025 Availability Groups

bit Obscene: SQL Server 2025 Availability Groups


Video Summary

In this video, I delve into the exciting world of SQL Server 2025 availability groups, discussing enhancements and improvements that address common pain points for DBAs and infrastructure teams. We explore key features such as improved monitoring tools, more discoverable settings, and better support for readable secondaries with query store data. Additionally, we touch on regular backup capabilities on secondary replicas, which significantly reduces recovery point objectives (RPO) issues. The discussion also covers the availability of various configuration options scattered across different management interfaces, highlighting areas where Microsoft could improve user experience through a more unified approach. Overall, this episode aims to provide insights into what SQL Server 2025 has in store for availability groups and how it can better serve the needs of the modern database environment.

Full Transcript

Yeah. Welcome back to another exciting episode of the Bit Obscene radio program. Since Sean has to go donate blood to his gator farm in Florida in 15 minutes, we are going to talk about 2025 availability groups so that Sean can showcase his expertise in that area and then go and do whatever he needs to do with his reptiles. So, Sean, I guess we’re picking on you first two in a row here. Since Joe has nothing to contribute, he’s just going to sit there and blink. We are going to, this is going to be the Sean show, you’re going to be elevate your temporary potential guest candidate co-host status to Olympian heights. So go ahead and take it away. And be a Greek Olympian? You can be whatever kind of Olympian you want. Right. Yeah, no, I mean, I think we’ve all had some interactions with AGs, even Joe. Right. Yeah, I mean, I had a bunch of latency problems, which was totally a SQL Server problem until it ended up not being one. Until someone, we won’t name names answered and found that it was the network. So what I want to know, Sean, is the SQL Server 2025 move data over the network faster. Have you personally made any improvements to get better network performance?

Sean O’ Yeah, sort of. I personally, so my main issue with what the crux of that with the latency and everything is I think the underlying messaging layer needs redone. Right. I think with some changes there, things will get better. Having said that, there were a couple things put in. So, you know, you all like fast, high throughput machines, right? Sean O’ Yeah. You like that. That’s good queries. That’s sweet, sweet, fast queries. Sean O’ Firehose and cursors. That’s not about the firehose.

Sean O’ So there’s a couple things in there. Have either of you heard of Group Commit? Sean O’ I’ve seen the wait type for it. I haven’t. I don’t think I’ve ever experienced it doing anything magnificent for me. Sean O’ Isn’t that like what we did when the three of us agreed to talk about this?

Sean O’ Yes. Yeah. Sean O’ But I think there are probably fewer razor blades involved. Sean O’ Okay. Sean O’ They’re missing out on the fun. Sean O’ Yeah.

Sean O’ So yeah. So group commit, I mean, essentially is when you go to commit, can we package up? Can we hold off writing that to disk? I mean, this isn’t an AG thing. It exists outside of it. But can we hold off writing that log block? Because commits and log blocks, they’re one of the things, right? You can hit the maximum size, which is 60k. And then that ends a log block where you get a commit and that ends a log block. Sean O’ So if we can put multiple commits in one log block, then we get more efficiency, less on disk space, blah, blah, blah, blah, blah, blah, right? You get one right to the IO instead of 15, whatever. So if we, if you have a commit that’s going to happen, you know, you do insert, insert into Joe, you know, salary, $5. And then you go ahead and commit that. And you do that. You have all these little tiny commits. Sean O’ I know, right? Zero dollars for Joe. So you have a bunch of those little small, your typical LTP style, you know, you can batch all those commits up and then write them as one, which means if you’re using something like an AG, then you’re only sending one log block across the wire versus 50.

Sean O’ So if you do have a latent, latent pipe like Joe has, sorry, I didn’t mean to throw out your medical issues on online. But if you have a latent pipe like Joe, then that can be very helpful. Sean O’ But then it’s not super great because you sit here and go, well, I don’t, you know, I have a fast, I’m using a hundred gigabit, you know, DAX with all this stuff and you’re clearly not running in the cloud. So you actually have sub millisecond latency. That might not be good. You might not want to hold up for five milliseconds, right? While you send it because it would have been faster for you to just send it.

Sean O’ You would have got it there and back faster than waiting for the group commit. So the group commit wait time was added in. So you can kind of remove that. That’s just a set option. I think that’s really good. One of the things that people I, when I’ve helped people and they complain and we talk about box cars and some other stuff, which you don’t really have to know about, but there’s a limit to the amount of outstanding log, right? Sean O’ Because you don’t want to say, you know, it’s not log shipping where you say, okay, great. Do a log back up and then ship the log over. And that’s whatever the outstanding is. You know, this is how much, how much do you want to send it without being acknowledged? Sean O’ Right. So that’s your outstanding. That’s your, your commit. And so that’s your number of box cars. So that’s a slightly configurable now, just there. And again, a, a, a SP configure option. And most people are going to think bigger number better as per usual, not always bigger number better.

Sean O’ It’s arguably slightly more, but you don’t, it’s like a parallelism and queries, right? If, if one thread’s good, eight threads, clearly better. Clearly won’t run slower. There’s no other issues that would ever it’s linear. Sean O’ Yeah. Why not? Sean O’ Eight times throughput.

Sean O’ Yeah. I mean, let’s just roll with that. Cause you know, Sean O’ Exactly. Sean O’ Every time I’ve been like, you know, forced to parallel plan, I’ve been really happy with the results. So yeah. Sean O’ I mean, I don’t even know why there are serial plans. Everything should just, even if one row is coming back, if you run it maxed up eight, it should just cut that row up into eight individual sections, run them individual faster and come through. Right.

Sean O’ It should, what it should do is run that query, run that across eight threads and whichever one finishes first wins. Sean O’ There’s something called batch mode aggregation, or at least I think that’s what it’s called. And you might end up with a count or a total. Sean O’ Oh, no.

Sean O’ And you might end up with a count. And parallelism does indeed make it faster. Sean O’ Sounds like someone who likes residual predicates, but we’ll move on. Sean O’ Joe likes all sorts of strange things.

Sean O’ This is 2008. You know, for whatever reason, the internet, like it can never advance past 2008. Sean O’ Like R2 SQL Server knowledge. You know what I’m talking about. Like it’s just impossible to advance past those. Sean O’ Yeah, but the other, some of the other things, have you, you guys aren’t, I know Joe doesn’t like AGs all that much, but contained AGs. So you got your logins, your jobs. In the AG, you got replicated master, replicated MSDB. So they come over.

Sean O’ I think those are great. Because, you know, a lot of what I see people struggle with, with AGs is keeping all that stuff synchronized in a meaningful way. Sean O’ So, you know, Sean O’ 90% of the reason they use DBA tools.

Sean O’ Yeah, exactly. Sean O’ Sync the logins and jobs. Sean O’ Exactly. You know, it’s like, and then like, you know, like, you know, of course, adding a step so that the job doesn’t run if the, you know, if it’s not on the writable or not on the primary or whatever. So, you know, like, you know, there’s just a lot of stuff that goes on with those. You’re like, I wish this was, I wish this didn’t suck. Sean O’ Yeah, so one of the big drawbacks for contained AGs was it couldn’t be in a distributed AG. Now it can be so too many kinds of AGs now.

Sean O’ There are. Yeah, we didn’t even talk about Linux. Sean O’ We’re not gonna. Sean O’ That’s the correct answer.

Sean O’ Just like we weren’t going to talk about Express Edition and the standard edition video, but someone, someone had to go say the word. Sean O’ Well, here, here’s one Joe like, yep. Sean O’ To me to express my name.

Sean O’ Well, you’ve been excited. Sean O’ How do you feel about query store on readable secondaries? Sean O’ I mean, I remember when it was announced and it was supposed to be it was in preview for a very long time. And then it was maybe generally available kind of but not supported still. And so, you know, I’m cautiously optimistic that they’ve worked out all the kinks and bugs with that. So that’ll be that’ll be nice. Because you know, the plan cash is terrible.

Sean O’ Never let that optimism fade. Sean O’ Yeah, never let them take that. Sean O’ I’m curious about that. So before we had that, like, like, what were the options for getting your query diagnostic data? Was it only plan cash?

Sean O’ Yeah, pretty much. Yeah, because query store data just shipped from the primary over. Sean O’ 2008 R2, man. That’s, that’s, that’s, that’s all you need. Sean O’ Yeah.

Sean O’ For tuning by plan cash. Sean O’ Yeah, just the plan cash and some logical reads and you’re just good to go. Sean O’ Yeah, and plus the fact that whether I don’t know if you know this, but those, you know, will come over to the primary. So you have that. Sean O’ It’s like merged replication.

Sean O’ I know, right? Sean O’ I still know some people who like that barf. Sean O’ But yeah, I think there’s a lot of great things for 2025. In terms of availability groups. I think it really hits a lot of the previous issues that you would have, especially, you know, between DBA, who’s typically in charge of it, right? Or infrastructure people, some Sean O’ sysadmin who’s like, All right, I got to set this up. I got to do this. No, we can’t do it. Can’t run into a lot of things. I still don’t think items are as discoverable as as they could be. Obviously, management studio doesn’t have support for a lot of things distributed AGs. And there, I shouldn’t say support. There’s no nice GUI wizard.

Sean O’ Yeah, no, nice. Sean O’ I think the biggest thing right now for people and the thing that I probably get the most questions on, even on stack exchanges. You know, how do we monitor this? How do we monitor this? Sean O’ How do I know there’s a problem? And where do I know there’s a problem?

Sean O’ Nothing in that has been changed. Sean O’ No. Well, that’s because Microsoft really does hate the GUI. Or rather, what I should say is Microsoft hates the end user and they express that hatred through various GUIs. The query store GUI being one of them, the AG dashboard being another. There’s a lot of stuff. Sean O’ You’re just like, like, I wouldn’t treat someone who I like and respect in this way. Like I wouldn’t do this to someone who I was a fan of like, if I was just like, hey, that Joe and Sean, they’re, they’re nice people.

Sean O’ Here’s how I’m going to let them look at their data. I like it just wouldn’t happen that would not go down that way. Sean O’ So I want to know what’s been I committed in a former life to be forced to use the SSMS extended events GUI. Sean O’ That’s another one.

Sean O’ It must have been a really bad one. It must have been a really horrible thing that I did. Sean O’ You used my secret. Sean O’ I would like to apologize to the SQL Server community for my karma being gripped upon all of you. Sean O’ The last two things I really want to highlight here for the 2025 improves on AGs when we were saying about, you know, things being given people heartburn, you can take regular backups on secondaries.

Sean O’ So you can take a full backup. Sean O’ Oh, not the regular anymore, right? Sean O’ Not the cop alone.

Sean O’ So it’ll change your, your diff map now will clear diff maps and your ML maps and stuff. Sean O’ That’s nice. Sean O’ You know, my, my, my gripe with the, like the set, but taking backups from the secondary was always like, Sean O’ They’re like, like there was nothing that would sort of warn you if the backup you were taking was far behind the primary.

Sean O’ Right. Sean O’ And so like, you know, like you’re sitting there, you’re taking log backups every minute of the secondary. Sean O’ But meanwhile, the secondary is like five, 10 minutes behind the primary and you’re like, well, like there, there goes my RPO.

Sean O’ Like, like, like, like, I wish there was something a little bit smarter with that. Sean O’ Like, you know, if someone’s like, you know, sets up all their backup jobs, so they go off a secondary because I don’t know, like, like, like, whatever reason people, people get really worked up. Sean O’ They’re like, Oh, the primary can’t handle the backups.

Sean O’ I’m like, what are you doing with your primary? Sean O’ But like, it would be nice if there were something sort of intelligent in there that would be like, wait a minute, like, like, this is pretty far behind that. Sean O’ Maybe we shouldn’t take backups here for a little bit.

Sean O’ Maybe we should focus on the backups here instead, because like, you know, you would you would you would have less chance of losing data in that scenario. Sean O’ But anyway, go on. Sean O’ Yeah, no, those I think those are the big things.

Sean O’ You know, you get some of the stuff with the readable, the query store. Sean O’ Secondary should get the backups. Sean O’ I mean, these are all top, top complained about items from everyone.

Sean O’ And I don’t mean that in a bad way. Sean O’ I mean, that in a good way like that. Sean O’ That’s a thing that I feel it should have been there from the beginning. Sean O’ Yeah, you took customer feedback and you implemented it.

Sean O’ And well, I didn’t. Sean O’ But yes, well, someone did. Sean O’ I mean, we all know you have source code access and you could really be a lot more helpful into these conversations. Sean O’ But, you know, we do it.

Sean O’ We do appreciate the contributions you do make. Sean O’ So thank you for that round of applause for Sean. Sean O’ I don’t have a round of applause sound effect. Sean O’ Oh, you know, we’re just going to insert.

Sean O’ A silent round of applause for Sean and all his. Sean O’ There’s the one in the zoom. Sean O’ Oh, there’s a single hand go. Sean O’ Yeah, no, I just think there’s a lot of really good stuff. Sean O’ I don’t like that.

Sean O’ A lot of these, you know, we’re talking about sequence. Sean O’ I know, I know I don’t like to know a lot of these things are in different places, right? Sean O’ We have not to segue into other stuff, but we have database scoped items.

Sean O’ We have SP configure or we have alter server. Sean O’ We have all kinds of stuff. Sean O’ Like, let’s pick one.

Sean O’ Yeah, right. Sean O’ Do you guys remember when sack was a thing? Sean O’ That’s the 2005. Sean O’ What is it? Sean O’ Well, it was, it was called wasn’t it called sack?

Sean O’ It was like the server administration. Sean O’ Oh, yeah, yeah, yeah, yeah. Sean O’ And you could export the XML or whatever and import it and do the, it was the way all the security things where you had to run a different.

Sean O’ Oh, God, that was mine. Sean O’ Yeah, well, thankfully, we moved on past that. Sean O’ Anyway, it is it is 430. Sean O’ And Sean has to go shave the virgins so that there’s gators don’t get heartburn again. Sean O’ So we’re going to let Sean go get to whatever he has to do and then some small chicken Joe Joe was me two episodes Sean O’ Because he’s been useless on this one.

Sean O’ So me and Joe. Sean O’ I think I show you one episode. Sean O’ How’s that two to one work? Sean O’ Because you were twice as useless on this one as you are usually. Sean O’ Wow.

Sean O’ Yeah, you got you have you have some labor some tax stop. Sean O’ You know, I’m making the same face right now that you made when when I have Googled best consultants and I didn’t include your name. Sean O’ Oh, yeah.

Sean O’ I think I think I think you just intentionally left my name out of that. Sean O’ I think you just made a list of names that you knew would make me make that face. Sean O’ And and just and just read that only I don’t even think you’ve looked it up. Sean O’ I think you just pretended to type.

Sean O’ You came up with a list of people who, you know, are worse consultants. Sean O’ I could share my screen right now and we could do a live demo. Sean O’ No, that’s okay, because you would you would have you would have pre cooked this. Sean O’ You would have this is something that you wouldn’t.

Sean O’ Yeah, this is something that you would have pre cooked just to just to just to make get me to make the Sean O’ That face again. Sean O’ So anyway, Sean’s gonna go do his thing. Sean, thank you for joining us for I think three episodes of the bit obscene radio program.

Sean O’ And we do we do anticipate having you back for future episodes to prove your worthiness as a co host. Sean O’ So. Sean O’ So that would be great that we will see you in the future.

Sean O’ Otherwise, it’s time for Joe’s punishment. Sean O’ So goodbye. Sean O’ We’re justwolf Jesus.

Going Further


If this is the kind of SQL Server stuff you love learning about, you’ll love my training. I’m offering a 25% discount to my blog readers if you click from here. I’m also available for consulting if you just don’t have time for that, and need to solve database performance problems quickly. You can also get a quick, low cost health check with no phone time required.

Updating Video Posts With Summaries and Transcripts

Complaint Department


In case you haven’t noticed I haven’t been writing blog posts lately. Mostly I’ve been writing training material, and boy are my fingers tired.

Since the training material is getting recorded, making videos of stuff usually makes for a better practice run.

People love complaining about free stuff. Like Americans in Europe when they realize you have to ask for ice in your water.

Anyway, a step I have always been bad at has been providing a written summary of what’s in the video. It’s a step I can never bring myself to take after I’ve spent who knows how long writing and recording and thinking about the zillion other things I have to do with myself. It’s also not something that comes very naturally to me.

I hate doing it. I hate paperwork. I hate checkboxes. Why do I need to do this? The video title is self-documenting.

Pumping Irony


So, hey, LLMs exist. LLMs can do things that I don’t have the mental energy to get set up to do.

I’ve been putting Claude Code to work on a bunch of stuff that would… Let’s be honest, it would never happen if I had to learn all the stuff around the perimeter of what I want to do. It’s a bit like the cloud in some ways. What used to be a rather monumental effort if you wanted to try to build something: domains, database servers, app servers, firewalls, and so on. All the stuff that would be a real impediment kinda became a click-a-few-buttons, spend-a-few-bucks, you’re free to do the thing you want now effort.

Where does the irony come in? Well, I got it in my head that I could have ol’ Claude build me a process that would:

  • Download batches of YouTube videos
  • Use local LLMs to generate summaries, chapters, and transcripts
  • Automatically update my blog posts and YouTube videos with the appropriate text

I think right now, it’s mostly working. I’m starting with old videos so mistakes aren’t as obvious and public.

Back to the irony: The very thing that allows me to do this is probably also the thing that makes it far too late to do any of this, ha ha ha.

You’re all spare parts. Don’t forget it.

Along The Way


The idea started off simply enough, and to be honest most of it just kinda worked.

Pythonista
Pythonista
  • YouTube and WordPress have APIs
  • Downloading videos is easy
  • Local ollama server makes running LLMs on my M3 simple
  • The LLMs I’m using (whisper for transcripts, qwen for summaries) don’t cost anything
  • It’s mostly just a matter of writing stuff to logging files and working off that

Probably the only part that made life interesting was WordPress. Sometimes the semantic search for YouTube video name > WordPress blog post name wasn’t very confident. Sometimes it would find multiple posts with similar names. We had to expand the search to narrow the search, by falling back to (gasp!) keyword searches for YouTube URLs in post bodies, and then even further to what sort of link was used. Text links are different from embed links, after all.

So how’s it going now? Well, it takes some number of hours (depending on video length) to download and process the audio tracks. The YouTube updates are easy, it’s just one URL to the other. I have to manually check WordPress for some things before applying those, but that’s not so awful. It’s just reviewing a control file.

Sure, there have been some bugs and hiccups along the way. For example, refining the post matching: some posts got updated with summaries and transcripts that shouldn’t have. There’s also a weird thing happening where some posts will get the YouTube URL copied in a second time, but that doesn’t happen all the time. Anyway, I guess my point is: Yes, it’s imperfect, but it wouldn’t exist and I wouldn’t be doing this if I couldn’t have a robot do the monkey work. It does not take a tremendous amount of self-reflection for me to admit that if I had to do this myself, there would be a whole hell of a lot more bugs and hiccups.

After all, I’m a visionary. I’m a big picture kind of guy. I know where I want things to get to, but I’ll be darned if I can write a lick of python or C# or whatever else to do it. This probably shows in my stored procedures in a lot of places, too. I am not the most gifted logical thinker. I’m forgetful. Sometimes I’m in a rush. Sometimes I have one thought and get distracted by seeing a million things that I need to correct, do better, or forget how it works along the way.

A million rabbit holes later, I can barely remember the thing I was supposed to be doing.

For Your Enjoyment


What you should see over the next days/weeks is old blog posts with videos getting updates to have summaries and transcripts, YouTube videos get updated to have summaries and chapters, and then finally, blessedly, to the great delight of Kevin Feasel (and potentially only Kevin Feasel), new videos having all this stuff when they publish.

Thanks for reading.

Going Further


If this is the kind of SQL Server stuff you love learning about, you’ll love my training. I’m offering a 25% discount to my blog readers if you click from here. I’m also available for consulting if you just don’t have time for that, and need to solve database performance problems quickly. You can also get a quick, low cost health check with no phone time required.

bit Obscene: SQL Server 2025 Standard Edition

bit Obscene: SQL Server 2025 Standard Edition


Video Summary

In this video, I delve into the latest updates for SQL Server 2025 Standard Edition, discussing its key features and limitations alongside my co-host Sean. We explore how the increased memory and CPU limits can benefit many users without the need to jump to Enterprise Edition, while also highlighting some areas where Standard Edition remains somewhat restricted, particularly in columnstore indexing and certain performance optimizations. We touch on practical considerations for those considering a migration from older versions or Enterprise Editions, as well as the potential benefits of moving to managed instances or Azure SQL Database. The discussion covers a range of topics, including High Availability Group (AG) basics and the challenges they present compared to more advanced features available in Enterprise Edition. Throughout the video, we aim to provide insights that can help database administrators and IT professionals make informed decisions about their SQL Server environments.

Full Transcript

Erikodynamovsky, L- Welcome back to our presumably second 15 minute episode of the Bit Obscene radio program. We’ll be talking about standard edition for SQL Server 2025, this go around. I do want to make a quick announcement here that this beautiful Darling Data T-shirt, I wore this during one of the days of the pre-con that Kendra Little and I did at Past Data Community Summit. in 2025, or well, I guess a few weeks ago now, and I will be auctioning this off. The proceeds of all of the auctioning will go to the Darling Data Home for Little Wanderers. So we do hope that this shirt will sell for quite a bit of money. It’s still… Did you wash it? No, it’s unwashed. It still smells like pre-con smear. Fresh Erik smell. Yeah, so you got a whole day of me in a shirt, on a shirt. So, hopefully, that will increase the price. Anyway, welcome back. We’re going to kick this off by… Let’s start with Sean this time, because Sean was…

Oh, baby. Sean was blissfully quiet during the IQP talk. So we’re gonna pick on Sean and make him talk about standard edition of SQL Server 2025 here. Sean, take it away. What is your opening salvo? Sean, take it away. I think there’s two things to talk about here, and I think they’re both good, right? You got the standard developer edition. Correct. Which technically meets standard edition requirements of this talk. And two is the increased limits for memory and CPU. I don’t see either of these as a negative in any way, other than more people complaining that it doesn’t have the features of Enterprise Edition.

But, you know, I mean, you opted to up the memory, 256 gig, which… Still pretty good, especially if you look at cloud offerings. Yep. Go see our talk on cloud. And, you know, given how much RAM costs now, right?

Oh, yeah, with the recent… No, it’s crazy. I bought RAM for 200 bucks a year ago, and now it’s like 700. Yeah. Wow.

What should I auction my laptop off? I have 128 gigs in my laptop. I feel like I should auction it with a micron. The brick of gold. Yeah, right? Yeah. Nice. If any of it’s micron, you might be able to get sentimental stuff now since they’re exiting the market.

I don’t actually know what’s in there. I don’t know. I could… If I thought either of my kids were destined for college, this would be their college fund. I was gonna say at this point.

Yeah. I mean, I have points in a liquor store. But, aside from that, I don’t know. Some of them are… The lesser of four sockets. Some of them are e-cores, so we don’t really count those.

Good. The lesser of four sockets are 32 P-cores. I agree with Sean. Everything is the lesser of four sockets. I agree with Sean. It seems like Microsoft did a good thing for once, and it’s hard to even complain about it.

I’m speechless. Joe agreed with me. Is that allowed in his contract? Fully agreed.

Well, the reason I don’t agree is because normally you say things that are wrong, but now you said something that was right. So I agree with you. It’s simple. One thing you mentioned is the resource governor edition on Staron, which could be huge for certain workloads. Memory grants specifically.

Yeah, using Comstore. It’s really easy to get these ridiculous 25% memory grants, which you might not want. I’m a big believer in resource governor for Comstore. I don’t know how you use it without it, but apparently.

Just not for CPU. No. Yes. Yes. Not for CPU. That’s correct. For memory and for maybe tracking parts of the workload to do reporting on it, but not CPU or CPU limiting or that stuff.

Yeah. People touch CPU and to go from an earlier joke, you know, if you just configure it correctly, all these problems go away. So.

So if you just configure it correctly, our friend Forrest has a good blog post about resource governor and CPU, which I would. I would argue people should read before ever touching it. There should be like a gooey checkbox in management studio where you go to check the box and you have to take like maybe a 10 question quiz.

Yeah. On it. Got to go read force blog posts and then take quiz. There are a lot of options that should require some quizzing before changing.

Like you should have to like, like maybe, maybe like a sobriety test. Like you should have to like. Well, I’m crazy.

Yeah. I never use management studio again. That’s bad. ADS is going away. That’s true. You’ll never, never catch me in there. But no, I think, I think Microsoft finally sort of like caught up with stuff. You know, for years I looked at developer edition is like a gateway drug to enterprise edition because people would make all their stuff on it and then roll out standard edition and be like, ha ha, we’ve gotten away with it.

And then standard edition just had so many crappy things about it that, you know, they need to run to enterprise edition pretty quickly. But I mean, 2016 added a lot of the items into standard edition. Well, yeah.

I think those were good things. They know that they were good things. And I’m not, I’m not arguing with that. But from my perspective, where standard edition is still woefully hobbled is around columnstore where everything is limited to adopt of two. Yeah.

And like, you know, granted you like there are some situations where, you know, batch mode adopt two is still better than row mode adopt eight. But that like those are those are even those are a little bit tricky at times. So I like, you know, if they like I think that, you know, the next thing would really have to be opening up some of the perf stuff to be cross edition, because some of it is and it gets confusing to figure out like, like scalar UDF inlining is enterprise and standard.

And it’s like there’s like there’s some stuff where it’s just like, no, it’s a net positive for society. We should do this. And then there’s other things are like, no, that’s enterprise only.

And I just feel like hobbling batch mode on standard edition is one of those like, like, like mean, like robber baron. It’s just like your family starves this Christmas. You didn’t pitch enough coal for me.

It’s like, ah, crap. Like, so that that that to me, like, you know, there’s still stuff about standard edition where I’m like, nah, like, don’t you can’t use that. But I do think Microsoft did did some good deeds with it for 2025. And I’m going to try not to gripe too much.

Eric wants something to complain about. So well done. Yeah. Like the batch mode and Magstop 2 thing. I forgot about that. Like, you know, they’re going to finally have like a brand new paradigm for parallel processing. It’s going to be super good.

It’s going to scale super high. It’s going to solve so many of the problems with Roma parallelism, you know, skewed rows and the in the repartition. With my own customization streams, very busy schedulers.

With the remote query, but you can only ride a Magstop 2. It’s like giving you the smallest possible taste of a drug. That’s how you get people, right?

The first one’s free. Small taste. Just start. I mean, I guess so. Yeah, that is an interesting limitation. Well, that’s not true.

I mean, standard edition a little bit in Pride, but I wouldn’t call myself a standard edition. I mean, standard edition is great for a lot of people, though. It’s just so express.

Well, that’s what I’m saying. Like, 2025 made Express even better. I agree. Agreed. But standard edition solves a really, again, a really unique area where Express isn’t quite enough. But Enterprise is obviously ridiculous in terms of getting a license and running it.

I think standard really does fit that role very well, especially with some of the security items that came through in the previous versions and stuff like that. But honestly, there’s a lot of things that people could run with standard edition that they don’t. And, you know, they use Enterprise.

And conversely, I think there’s a lot of things that people run on Express edition. And it’s like, I want to smack you. Stop. Just go to standard. Like, ah.

You know, especially now that standard edition can use, what is it, 32 cores? Yeah. Like, come on, man. Give me some better DOP options on batch mode. I want.

Well, if you think about the hardware, too, outside of, I know everyone’s getting super hot and heavy over ARM, which I’m going to argue is not the correct microprocessor architecture to run database stuff on. Even, you know, the ARM 64 EC stuff and whatever. But I’m sure I’ll get roasted for that.

It’s fine. I don’t really care because it’s, you know, you’re wrong anyway. But the amount of CPUs, at least for, like you were saying previously, you know, you had your laptop had some E cores in it. The amount of CPUs that are x86, 64, and our full execution units hasn’t really gone up super high.

Right? Especially not ones where if you want to take, you know, you’re talking about columnstore, if you really want to take advantage of a lot of that stuff, and even a lot of the new features where you’re getting crazy efficiencies from. I mean, you want those SIMD instructions, right?

You want AVX. You want all that stuff. You’re not going to get that on these crappy ARM and these crappy other CPUs or E cores or whatever. So, I mean, 32 cores, I think, is a really good spot, even by today’s standards, for actual decent execution of stuff with having full processor capability and what that entails.

Sean, I’m curious. Do you think some standard edition customers end up kind of being blocked from doing a migration to, like, managed instances or Azure SQL database just because they’re getting so much savings and licensing costs that going into a cloud platform that eliminates the difference between standard and enterprise just doesn’t make sense for them? No, because typically those people are going to go to that anyway because their CIO told them that we need to be cloud first.

Well, but in the maybe rare case where the CIO wants to report of, well, we’re paying X today, we’d be paying Y on the cloud. You know, if you’re standard, that’s always going to look better for you in terms of comparison, unless you’re doing, like, Azure VMs or something and you still stay in standard. Yeah, I wouldn’t say it would.

My personal opinion, I don’t think it’s a blocker because you’re already running standard locally unless you’re really, you’re going to be blocked from migrating for some other reason, right? Like, you can’t put your data there or whatever. And if you are going to migrate, it’s going to be outside of, you know, idiot telling someone that we have to be there.

You’re going to do it for other reasons, such as like, you know, we want more controller or longer backups. We don’t have the infrastructure for it or we really don’t need 64 cores, right? Or 32 cores.

We really only need four and we actually may save money by doing it because we only need a four core thing on MI or hyperscale or something. So I don’t think it’s really blocking anyone from doing it. I think that’s obviously nice that you get the enhancements, but I don’t think many of those people are doing it for the enhancements.

Like, I don’t think it’s the standard having new stuff is really going to block any of that. If you were going to do it, you’re going to do it first. If you have a legitimate reason, you were going to do it anyway.

If you don’t have a legitimate reason, you were going to do it anyway. So let’s suppose you’re on 2008 R2, which we all know is the best version of SQL Server. You’re on Enterprise and you don’t have SA.

Well, I suppose that’d be obvious if you’re on 2008 R2. What are the big things to look out for if you’re considering switching from Enterprise to Standard other than the Comstar, which of course you’re not using because you’re on 2008 R2? Yeah, going from an earlier version like that, a lot of things to consider that you are actually going to be upscaled.

So what was in 2008 R2 Enterprise is just going to be in Standard Edition today, which is great. I mean, that’s awesome, right? You’re actually people need to upgrade, you know, get off of Windows XP and, you know, anyway, there’s a lot of good stuff.

I think the things actually to watch watch out for are going to be some of the stuff we talked previously around, like IQP and things of that nature, where your workload’s not going to run the same. I mean, you’re just not, you’re going to get stuff that was slow is going to be fast, stuff that was fast is going to be slow, and stuff that was slow is going to stay slow. It’s going to be a mixed bag.

Like there’s, that’s really the biggest thing. Good. We’re talking about a more modern version of SQL Server, right? Like from 2019 or so, because the IQP stuff is pretty recent. So if like, yeah, but think about it, don’t have SA, it’s free to upgrade.

Is that how that works? Yeah. If you have SA, it’s free to upgrade. If you don’t, you don’t. So I don’t know if there’s been enough time where like, if you don’t have SA, you’d be looking for an upgrade. But look how many places are running in 2016, which is end of life, right?

I mean, there’s so many places around 2016 where a lot of those, I know you prefer not to. That’s the, you don’t want to look at those skeletons. We definitely didn’t retire a SQL Server 2016 instance last week.

That definitely didn’t happen. That’s not something that happened. I’m just going to say that for the record. What do you think about the basic AGs?

So let’s say I’m a very cost conscious customer. I’m going to use standard. Like what, what’s the starting point for HADR? Yeah.

The starting point is definitely. So technically database mirroring is still in there, right? You can still technically use it, but I will slap anyone who does. Like I will find you and I will slap you. Some people are into that.

I, well, it might be slap as a service for. Then you’ll get an invoice for it. Yeah. But the it’s, that’s actually a really great question, Joe, because it’s, it does offer the basics.

Like I’m not, I’m personally not a fan of basic AGs. Just in general, you get, you know, to what you were saying before you get just enough where it’s sort of helpful, but not really because of how it’s implemented. Like you get that taste and yes, that’s, that could be an impetus to upgrade.

But if you need that level, you should have been an enterprise anyway. I guess that’s just an argument there, but for the basic AGs, I do. I love them in terms of you actually get decent functionality for it.

I just hate that you can’t, there’s no upgrade path. And what I mean by that is you put in the basic AG. Yeah.

You can’t do your readable secondary. You can’t do a bunch of stuff for the reason why you would actually use it. Uh, but if you’re saying just for HADR, you kind of get the HADR, right? You kind of get it.

It’s not a hundred percent, but you, you get 50% of the way there and, and you can play with it and say, okay, this may actually work for the, so for this one rope, you know, maybe critical system or something, that’s where we go. We don’t upgrade any of the other licenses.

Sure. But I don’t like that there is no upgrade path. So what I mean by that is let’s say you create a basic availability group and you say, this is, this is awesome.

We love it. We want to go to enterprise. We tried developer edition, you know, as a, as an example for enterprise and it worked really well based on what we’ve done. This and that.

Not now, what do you do now? You take a replica out upgraded to enterprise and do what? Yeah, exactly.

As someone who is very ill-informed about this, I did ask a question on Stack Exchange four years ago and I think you were still boycoring at the time because you didn’t answer personally. So you really let me down there.

So the question I asked was basically, I’m reading about basic availability groups. You can’t do integrity checks on the secondary. This seems really bad.

Like, is this even tenable? And the answers were generally, well, if you’re willing to like fail over often so you can integrity check your secondary, then sure. That’s what everyone wants to do.

Yeah. Fail over into an integrity check. Yeah, that’s definitely what everyone wants to do. But speaking of a taste, I’ve heard rumors that we’re going to have a video dedicated to how 2025 changes pages.

Have you heard that rumor, Eric? I have. I have. I’ve actually heard that rumor about someone on this radio program producing that video. Yeah.

That person should really get it done. Yeah. That person should get it done. Anyway, I have two thoughts on this, and we can close out whenever. One is it two.

I was giving you the perfect way to close out, man. Let it go. 2008 R2 is not the best version of SQL Server. 2008 is the best version of SQL Server. Because that was before Microsoft changed their licensing model.

So I have a client on SQL Server 2008 Standard Edition, because they have a lot of memory, and they do not pay per core. So they’re balling out there.

And the other thing is that I do not understand why anyone would go the route of a basic availability group when a failover cluster in log shipping still exists in the world. Yeah.

Yeah. Yeah. You’re on Standard Edition. Don’t bag it. Failover cluster in log shipping. Yeah. Failover cluster in log shipping. Sounds like you’re trying to answer it to my question. I heard a rumor that log shipping was updated in 2020.

I can mark it. Really? Accept it. Wait. Wait. Wait. Hold on. Stop the presses. What got updated in log shipping for 2025? It’s breaks when you upgrade, right?

Aside from that. Oh. Wait. It’s called a feature, Joe. Wait. Did you want more than that? Yeah. I wanted more ammo for my log shipping love. It supports TDS-8.

Oh. Well, that doesn’t do anything for me. I’m sorry. You’re more secure, Eric.

I was already secure. I was already quite secure. All right. Well, I think that’s enough about Standard Edition. Thank you all for watching. I don’t know what we’re going to talk about in the next video yet.

It is undecided, but from the look on Joe’s face, it will be something very serious. So whatever we decide to talk about, you will listen to and enjoy. Anyway, thank you.

Goodbye. Goodbye. Thank you.

Going Further


If this is the kind of SQL Server stuff you love learning about, you’ll love my training. I’m offering a 25% discount to my blog readers if you click from here. I’m also available for consulting if you just don’t have time for that, and need to solve database performance problems quickly. You can also get a quick, low cost health check with no phone time required.

bit Obscene: SQL Server 2025 IQP Features

bit Obscene: SQL Server 2025 IQP Features


Video Summary

In this video, I dive into the world of SQL Server 2025’s intelligent query processing, discussing it alongside my co-hosts Prop Joe Obish and Sean Gilardi. We explore topics like AI-driven cardinality estimates, which can sometimes lead to unpredictable performance issues, as well as the challenges with parameter-sensitive plans and abort query execution hints. We delve into how these features might impact real-world scenarios for DBAs, especially those working with third-party applications where they have limited control over queries and indexes. The conversation highlights both the potential benefits and the pitfalls of relying on such advanced features in SQL Server, offering insights from practical experience to theoretical considerations. Join us as we dissect these complex topics and share our thoughts on how Microsoft can better support DBAs in their daily tasks.

Full Transcript

This computer. All right, welcome back to the Bit Obscene radio program. I have two candidate temporary co-hosts with me today, Prop Joe Obish and Sean Gilardi. This episode of the Bit Obscene radio program is brought to you by Meridomo 100% Cacao, Sin Azucar, apparently. Brand new sponsor. They sent me this box. It’s lovely. You can eat it and your face gets a little numb and you feel a little sweaty and you just want to talk to people. So I like it. So I took a bite of that so I could talk to these two. Today we’re going to be talking about SQL Server. Actually, we’re going to do five episodes in a row. Short episodes, hit and runs about SQL Server 2025. The first topic is going to be, you can see the anticipation on Joe’s face. It’s going to be intelligent. Query processing. I don’t like how you committed us to doing five because now we have to do five. Whereas you had been like ambiguous, then we could have. The ambiguous number five. Well, we’re doing, we’re doing five. And if we do more, then they’re called bonus episodes.

Oh, we have to do, oh man. And if we do, if we do fewer, not less, we do fewer, Joe. Excuse me. Yeah. Okay. So have a little respect. If we do fewer, then it doesn’t matter because I will do two by myself and just pretend that I fired both of you. Okay. All right. So we’re good to go. Start us off, Joe. What do you, how do you feel about intelligent query processing, the state of intelligent query processing in SQL Server 2025? So the way I feel about it is a long time fans of the pod will know that I strongly believe in fixing a query performance issue once, and then you fix it forever and you never have to look at it. This is different from the consultant approach of wanting as many billblahers as possible, but I’m not a consultant. So I can get away with that. And, you know, the thing about intelligent is kind of means unpredictable.

Hmm. Right. So erratic even I’m going to show you my pets. So this is my, my pet rock. It is very predictable. It, it, it, uh, never poops anywhere. It shouldn’t. It never chews on wires. Very house trained. It is perfect for me. And this is how I want the Kray automation to be. I don’t want any surprises. I don’t want any. Very, very intelligent. Um, Oh, Joe, you were born just a little too late.

Changes. Right. Oracle have that. They called them. Roles. If you, uh, like if, like if we just get into the, um, well, you can have carnality estimates that change through AI or something, right? Like what does it even mean? Um, like I have a performance issue yesterday. I tried to reproduce it.

Today. I’m getting a new AI cardinality estimate. I can’t reproduce the issue or something’s working today. It gets worse in the future. Cause as we all know, all other than Sean, sometimes better estimates can lead to worse performance. That is true. I’m sure you’ve seen it. That is true.

So like with respect to the entire feature, uh, I guess, I guess, I guess it’s a bit of a, uh, design choice. Cause I remember someone, maybe it was, uh, Pablo Blanco told me that the, the, uh, legacy. See, there are a lot of like special cases, Microsoft developed over the years.

Presumably this was back when, uh, you know, connect actually worked. So, so people would, would complain about their issues and Microsoft would hear their complaints and fix it. And part of the, the new sea new being 11 years ago is it’s, they got rid of a lot of those special cases for, for better and worse. And I do wonder if this AI carnality estimate, like if that fits in that model or not, um, um, so to, to, to, to give an example of something I saw recently, as we all know, other, other than Sean, maybe if, if you’re filtering on a column that stores, like when the row was created, let’s say you do like, uh, creation time greater than get date, you expect there to be very few rows returned by that query.

Yeah. Yeah.

Yeah. Yeah. It was greater than get date at time zone.

Anything. Hmm. Suddenly that’s an unknown inequality. Pretty good. Hmm.

We don’t know anything about that. So now we’re in the, uh, 30% default guests situation. And I think the biggest Delta you can get for at time zone is 26 hours or so. Uh, you know, in real life, it’s probably going to be only a few hours.

Uh, so from the point of view of like, you know, I, I want. My pre-optimizer to know things like it could, it, it could constant fold at time zone, but it doesn’t. Or it could treat at time zone is not really being there.

Cause you’re generally shifting by a couple hours. Personally speaking, I wouldn’t expect to get a totally different Carnelli estimate just because I’m shifting a parameter by a couple hours. But like, that feels like the kind of thing, which would be, um, that might be a special case in legacy CE.

Maybe, I don’t know, but it’s certainly not in the new CE. And, you know, if, if we’re talking about all of this do AI stuff kind of feels like those types of fixes might be dead because like, why, why do you special cases when you can just have. Yeah.

I do everything. Right. I mean, in the, in the specific case of that time zone, I have never trusted that function in a query, uh, aside from maybe like, unless I need to, unless I’m using it, like in the select list to figure things out. Uh, most of the time, if I need to do anything with time zones, I’m not saying anything at time zone.

I’m figuring out like what the difference in hours is with that time zone and then doing date math based on that difference. Like I, which is probably more accurate to be honest. Cause yeah.

Cause yeah. Cause like then you can, you, you get, you constant full, like whatever that, that, that one scale our value and you’re good to go. So at time zone can screw itself. I hate that thing.

Uh, it ruined three of my nights when I was working on stuff in query store. So I don’t, I don’t even want to go there. Um, as far as like intelligent query processing, processing stuff in general goes, uh, you know, like I, I, I appreciate the, uh, the attempts and efforts to make things better, but it all just feels a bit bolted on at times. And it doesn’t feel, and it feels like all this stuff where it’s like, we’re going to do this heuristically and we’re going to like figure it out.

And like, you know, whatever, like, like, so like the two things that I have the biggest, uh, gripes with so far are, uh, batch mode on Roastor. Um, there is at least a newly discovered hint by our Russian friend who to override batch mode heuristics. So we have that going for us.

No, granted there are a million other ways to override batch mode heuristics anyway, but the details of that stink. But, um, the other one is the parameter sensitive plan optimization. Uh, the way that it, it gauges skewness and worthiness of kicking in is nonsense.

And the way that it buckets things is absolute garbage. And I, I wish that like anyone in the world with like half, uh, half a sense would, would take another look at how that, how that thing works and fix it. Because it makes no sense to have like a bunch of like middle values get bucketed because those middle values are not the highest value or the lowest value.

So like you’ll have stuff with like 10 million rows sharing a plan with stuff that has like a hundred thousand rows because they’re not the highest or the lowest. And I feel like that, that whole scheme is just the, one of the dumbest things that I’ve seen choice wise in SQL Server in a long time. Um, um, dot feedback is another one that belongs in the garbage bin.

Oh, wait, wait, hold on. What do you, how do you, why don’t we talk about brand of sensitive, whatever, as a group? Okay. Before, before we go into that, before we go into that. Okay.

Yeah. It seems like Joe, you’re, you’re almost asking, uh, for the old Oracle style rules where you say, join this 10% do, do a nested loop. Join always do this, always run it this way.

And you kind of lock it in because you know better. Oh no. That’s what it sounds like. So I’m asking for a way to be able to communicate possible query optimization improvements to a top Microsoft man like yourself. And then maybe there’s a small chance we get that in the product.

Um, like the, at the, at times, an example is entirely like, like theory based in terms of like, you know, like what’s reasonable or. Let me wear it this way. The more information you have during crowd designation, the better in general, right?

Generally at time zone, isn’t some scale or UDF black box. We have no idea what the output is going to be. It’s a very well-defined thing.

It has a well-defined range and how it’s going to change your data. If Microsoft wanted to, they could enhance, uh, constant folding or just. Kernel yes submission general to work better with that time zone.

Could. For sure. Now I’m not saying that, that, that, that they should, but it feels like, you know, like, like that type of tweak. Is maybe going in the wayside.

I’m not looking for anything like, like, like Oracle. Oracle is. Way too expensive. Oracle is its own set of problems. Um, so with respect to.

We can get to PSP, but I wanted to also get your feeling on abort query execution. Man, we’re just jumping around and I don’t have an opinion on that. Do you have an opinion on that?

All right, go ahead. Oh, I have, I have a strong one. Oh, go ahead. Oh man. Wait, we’re going to mix everything up and our poor radio viewers are going to be confused. Maybe we should go one at a time. So, uh, the, uh, the abort abort query thing is a major whiff by all of Microsoft, the entire organization, top to bottom.

Um, the fact that all it does is kill a query. If it starts running is absolute gobbledygook. Like that thing, if, if there were any, again, if there were any sense to anyone, then that thing would have a number that you could supply to it.

They would tell a query when to time out. Cause like most of the time, if you’re a DBA, you’re not like, it’s not like, oh, I never want this query to run. You’re like, this query has five seconds to finish.

And if it doesn’t finish, it’s over. Like, like just having a query get killed immediately is, I mean, I suppose there are some circumstances where you would put zero in and be like, no, you, you don’t get to run or whatever. But like, for the most part, you know, like this, like that, that hint is absolutely gorgeous for off the rails reporting tools like Looker and other crap like that.

Um, and like, you want to give those queries a chance, but if they, if they miss their window, then they, the query just gets executed. Um, there was something like that in resource governor, but you need a trace flag for it. It was like the max CPU time thing.

But like, unless you have a trace flag that doesn’t really pan out. And I just want to point out too, that the use, the use cases, not necessarily for a board query execution, but for a lot of this stuff can be, uh, you know, Joe and yourself are doing a lot of actual query tuning system tuning. You know, design, design, design help and work and stuff like that.

A lot of poor DBAs. And I, I was one for a long time, got handed very crappy applications bought by some third company, third party company that God knows what. And it said, you can’t touch it.

You can’t do anything. Also, we’re going to call you when everything breaks and run slow. Yep. And I’m like, what do you want me to do with it? I can’t, you’re telling me I can’t even, I can’t even make a, uh, you know, a plan override for any of these.

Like you’re, I can’t do a plan. I can’t do anything because I, you know, void our warranty on, on the app. So some of this stuff doesn’t make sense for people who are actually tuning queries, getting stuff done. But for those people, you know, have to deal with some of this crap that comes through and it, it, it’s not ideal, but it’s better than nothing.

Uh, I mean, sure. But like, like, I think Eric’s point is, it seems very similar in engineering complexity to have the feature as is compared to have the feature where he can like set a number of seconds. Because the feature as is, is effectively killing after zero seconds.

Maybe. And Eric just wanted to be able to tweak that one number. Is that accurate? That is correct. I, I think I can see both sides.

Like to Sean’s point, that was my interpretation of the use case too. Where like, if you find yourself implement, cause you could just do it yourself if you had to, right? Like, well, we have the query timeout, right?

You can set a query timeout, but it’s already run for that long and use those resources. Well, in the application you can, but like, yeah, well, I thought we were talking about you, you’re locked out of a third prior application. You can’t.

Oh yeah. Go ahead. Because I’m sure people have done things like I’ll throw an agent job up every minute. And if I see queries running longer than X seconds, if there’s a certain pattern, then I don’t know if we can kill those queries. So instead of doing that disaster, you could use this if you’re okay with just not letting them run out.

But you know, like that’s, that’s evaluating the feature from maybe too narrow of a use case. Whereas Eric is dreaming bigger and greener and with more rainbows. We’ll wash that.

I mean, my question here is how many, how many queries in, okay, let’s just say like, we’re in Sean’s world from when he had a real job. And like, you have these crappy third party applications, you’re not allowed to change a query and index a setting, anything like that. All you can do is like, I don’t know, like restart SQL Server if everything locks up.

Okay, there you are. And like, you know, you’re, you’re, you’re, you’re in that world, you’re in that place in time. And Microsoft is like, here’s this hint.

And you’re like, cool. Now I can just have these application queries not run. And now all of a sudden, you know, end user Bob and end user Sally are like, well, it’s time to go to work and do my job. And they, they click execute.

And it just comes back with an error. That’s like your query has been killed. Like, like, like what, like, what can you reasonably block an application that’s not going to mess up an end user’s day? Like, you have to give it a chance to run.

And if it can’t run, you say like your query timed out. That’s a much, that’s a, that’s like nice and friendly. If someone just like, you know, hits whatever button to like, you know, like, like submit something, and it hangs out for a little bit. And then like, it just immediately returns, like, you’re not allowed to run that query.

And they’re like, oh, cool. I can’t do my job. Who do I talk to now? Like, it’s just, it’s an, it’s, to me, it’s just wholly unrealistic to have that situation where you’re like, like this, like, and, like, unless, and look, and, and end users are annoying. Right?

Because like, you might have like a bunch of like, like business intelligence people who are going to be running stuff through management studio. And they can easily change like one or two little things about a query and have it completely bypass whatever you set up for that abort thing. So like, like, what’s the point?

Like, like, get, like, give the query a chance. If the query can’t finish within like, I don’t know, five, 10, 30 seconds, kill it off. Like, if it used the resources that caused a problem, it spiked things up for that time. Fine.

But then now it’s over. Right? Like, you’re done. I have an example for you actually from, from real life even. Oh boy. Like a developer messes up a query, which is like checking a queue. Uh huh.

And instead of running once per minute, it runs a billion times a day. Mm-hmm. Which is something I’ve seen. Yeah.

So like for that case, you don’t want your billion times per day query to even run for like, well, for like one second. Okay. Fine. Like, like, like, I would view it as like, you’re picking the lesser of two evils. Like, you know, the entire application is slow and down versus, well, we can turn off this, this queue thing for a couple hours and we can process the data manually later.

And then you just might not care that much about this thing, which is suddenly running a billion times a day when it wasn’t supposed to be. Okay. But if you’re, if one of your developers messes up a query like that and you turn off the queue service and you say, hey, developer, make a hot fix with a wait for delay in there.

Like, like, like to me, like, like, like, you’re not like, you’re probably not going to find an enterprise application that screws up that badly. Like, granted. Okay.

Like there, there are cracks that everything can fall through, but like for the most part, you’re not going to run into a situation where you’re just like this query can never run. So you’ve never used teams. No, you’re right.

I specifically avoid not using teams as in my consulting contract. So you’re effectively basically saying the same thing, right? Where you’re viewing the, the kill zeros at zero seconds thing as too narrow and in your, in your technicolor dreams, you want to be able to change that from zero to X. And that’s going to cover more use cases, which I don’t think anyone would disagree with.

But I mean, you know how it is. Everything Microsoft does, it’s never done right the first time, right? Yeah.

But the problem, the bigger problem for me is that there’s often not a second time with a lot of these things. So you’re just stuck with things in V1 and you’re like, cool. Oh, never going to use that now. Like just, perhaps we can add some intelligence to the development process for intelligent query processing.

Perhaps, perhaps we could. You could call it, I, I, Q, P. But, but how do you know, but how do you know how to intelligently fix a query if it’s not allowed to run?

Like, you know, you, you, you, you’ve got to give it a chance. You could even have like a whole thing. You could even have a whole thing that kicks in and it’s just like, like this query has been aborted because it didn’t finish in 10 seconds, like a hundred thousand times.

Like, let’s really think about this one. Let’s really do our best to make this one better. Speaking of doing our best.

Yeah. I believe we agreed to a specified length of the video and we’re too much over it. But with respect to parameter sniffing, your favorite thing in the world, my understanding of the, what’s this thing even called?

Parameter sniffing. Plan optimization. The parameter sensitive, parameter sensitive plan.

Oh, parameter. Yeah. Parameter sensitive plan optimization. Do you know if it still only works for equality predicates? Yes.

Only equality predicates. There, there are a number of strange rules. That one feels weird to me. Like maybe my world view is too narrow, but. I don’t think it is. You got the gloves on.

That’s why it feels weird. Most of the. More fake news from Microsoft. Right. Most of the. Parameter sensitive.

Yeah. I see are relating to inequality predicates. Yep. On dates. Where you know that they, they have passed in today or a year or, or a 10 years from now. Yep.

And it’s not so much. Equality predicates. Uh, at least that that’s in the workload that. The workloads that I know of. So the whole thing kind of feels. Not exciting.

Yeah. No, I mean, like in, in my, you know, consultant demo world, it’s very easy to find a quality predicate issues in the stack overflow database. You have like the post type ID and vote type ID stuff.

And there are wild swings in the number of post and vote type ID. So it’s very easy to show it a not working when it should, uh, with those and be the bad bucketing that happens even when it does work. Um, but I think generally you’re right in that, uh, you know, a lot of the problems that I see with, uh, parameter sensitivity are with, um, bounded date ranges where, you know, for the majority of the queries that come in, they’re looking at the last hour or day of data or something.

And, or like the current working day of data, whatever it is. And then every once in a while, someone will come along and say, oh, well, I need to see six months of crap now. And then that plan just, you know, bites it.

Yeah. So like, yeah. So like, I, I feel like there should be some, like, there should be something like rather Microsoft has enough crazy rules around date. Things in query plans, like get range through mismatch types and get range through convert that it feels like a very natural fit for the project.

Parameter sensitive plan optimization to have special rules around dates where it should be able to look at them and be like, oh, like, yeah. Like the last time this ran, it was for the last like four hours of data. And Hey, this person’s looking for a much bigger range.

We should probably figure something else out. Like that seems like a, like a good special case for me, but what do I know? Maybe part of the thing that makes it tricky is you often see, you know, the start date and end date parameters too.

So then it’s like, you have to optimize for the pair of parameters, which I’m sure is way too much. No, you only, you only, you only have to ref, you only have to engineer for the distance between them. Well, right.

But if your startup teacher has like a hundred parameters and you know, like you as the, as the expert know, well, these two parameters really matter the most. I, you know, like the problem has shifted from one equality predicate to one inequality predicate to the right pair of inequality predicates. It certainly seems possible, but I mean, I, I still would like them to do inequality first.

Yeah. You know, for, for me, it feels like the optimizer is smart enough to pick up on those patterns when a query is being like optimized. Like it, like it, it can, it, it like, it’s like looking at the parameters and it’s looking at the columns and it’s looking at the way that they’re being assessed.

And you can make a, it’s pretty easy to infer from all of the various trees and, you know, parsings and bindings and all the other stuff that goes on that like, oh, like, yeah, these, these two pair up together. Like, make sure if you just like saw like a store procedure and a list of parameters, it might not be obvious at first, even though you might have something called start date and might have something called end date. And maybe some queries use start date and some queries use end date and they don’t pair up like that fine.

Like, you know, there’s, there’s edge cases, but like, as soon as you start like really getting into looking at a query and trying to figure out a plan, you have a pretty good sense of when they are paired and when it’s probably a good idea to figure something out. Oh, Sean, Eric thinks it’s easy. Will you have good news for us in C++ Server 2021?

Or 2030 or whatever, whatever else. Yeah, whatever. When are they going to have local variable deferred compilation? No idea.

Probably never, because think about how many like behavior changes you get from that. Well, I know, but like it should be, the option, the option should be on the table. We’ll just do that.

Add that to the connection options. Yeah. Option. Eric’s additional option. It’s everything is just a database scope config or a query hand. There’s just, there’s probably.

Just look under preview features. Yeah. Speaking of preview features. No, we’re not going to switch topics. We won’t do that. We won’t do that to our nice listeners. We should switch topics.

You could throw a local variable into a one row table variable and then maybe get some deferred compiling on that. But you still don’t have a histogram on that.

Well, they shouldn’t prove that and add a histogram. Ah, well, then you just have a temp table by a different name. Yeah, but you have a temp table without all the temp table issues. But if it has a histogram, they just, well, I guess if they, they just don’t, if they don’t do the caching.

It’s not a single row, like it’s going to be no problem. Yeah, sure. There, right there. We figured it out. Okay. So table variables can have histograms as long as it’s on a single row. Right?

Sean, you’re taking notes. Yep. You see my list? Okay. Yeah. Yeah. I saw it. I saw it. How do we feel about DOP feedback? I actually had an opportunity back in 2019 when I was on Microsoft’s campus, a very intelligent gentleman who I respect greatly, but who also has no internet presence.

I’m not going to say his name. Asked me if I wanted to give feedback on the DOP feedback feature. I said, this seems not very useful.

You shouldn’t do it at all. Which was, I guess, not the feedback Microsoft wanted in that case. Because sadly, they still did it.

I looked at it when it was 2022 and thought it was pretty flawed. Should I just go for it? Yeah, go for it.

From what I remember… It’s a long silence. Well, it’s hard to figure out. I’m for it.

I don’t know how to word things sometimes. We’re reading from disk on this one. One of the big problems with… Azure pass. Row mode parallelism is you can really get hammered by performance depending on how busy the server is. Yeah.

Way more than batch mode, way more than serial queries. There’s a demo on a blog post on your site where I think the performance difference was something like 25 to 1. Is this the round robin?

Not round robin. It was the… Demand? Well, this is where demand would be way better. Ah, okay.

It was some very simple query with row mode repartition stream operators. And if you had like a single CPU that was very busy. Yep.

The query runtime got like 25 times worse. Yep. Due to the yielding. Mm-hmm. So I don’t know how you can effectively do DOP feedback when you’re not kind for things like the server is busy during these hours.

And this is when I really care about performance. Yep. Because the thing is gathering data all the time.

And if you start gathering data like after a server restart, maybe that’s at 2 a.m. Well, all your parallel queries are doing great at 2 a.m. Because there’s still little work on the server.

Yep. So there’s that big issue, which maybe it’s better in 2025. I have no idea.

I think they’re kind of cagey on the details. I don’t. The other thing that bugs me about it is I’ve never personally tuned a query to anything but max.op1 or really, you know, encouraging parallelism for query. Yeah.

Like I’m not doing, oh, will this query run at DOP2 and this one run at 4 or this one run at 6 and this one run at 7 because I’m feeling lucky that day. And like even worse for the poor soul who has to do that, this feature isn’t actually respecting that.

Because if you ask me how to design it, if we had to do it, I would say, well, if, you know, if, if, if a person or an AI added like, like the literal max.opx hint to the plan, presumably someone like looked at that query and like that DOP should not be overwritten by whatever automated process this thing is running. But that’s not how it works. I don’t know.

I don’t like it, Eric. I don’t like it either. I, I think it’s goofy because it only adjusts DOP down. It doesn’t adjust DOP up. It would never look at a query running at DOP 4 and be like, oh, DOP 8 is better, like magically.

But I think the, the, the, really the, the biggest flaw in it for me is that it will never adjust down to max.op1. It only adjust down to max.op2. And I remember hearing at some point that they couldn’t accomplish that without a recompile.

But we all know that’s nonsense because when a SQL Server comes under enough memory, enough CPU pressure, it will run a parallel query plan at DOP 1 internally. So it doesn’t actually need to recompile. And there’s really no reason for it not to get down to DOP 1.

But like, like for me, you know, the, the thing that I would much rather see happen is if like, if we’re talking about like parallelism feedback is like, like fix the parallel page supplier. Like if you have a, if you have a parallel query that’s ending up with incredibly lopsided threads, like, like you, like you have a DMV that shows that, that is, that is detectable by SQL Server. Like it, like you could actually do something meaningful with like whatever algorithm is being used to distribute rows to threads in order to balance that on like a future execution.

And that would be far more meaningful to me than like, you know, you know, some other features sitting in the background being like DOP 8, nope, DOP 6, nope, DOP 4, nope, DOP 2, mm, DOP 2, just leave it. Like it, it, it, I think it’s just a waste of time. You reminded me of another complaint I had, which is like, it seemed very focused on preventing recompiles.

But if we’re talking about tuning these big parallel queries, like I, I’ve never thought, oh man, I just parallel query that, that runs for 30 seconds. I have to avoid recompiling no matter what. Like, like that’s like, that’s my number one priority.

No recompiles. Um, that was split. Maybe that was just decided by, uh, cookies and it can’t be overturned, but. I don’t think cookies is on that one.

If you, if you’re able to move past that, I actually had a good idea. I think just now, like, what if we had a batch mode heuristic, like IQP where if, if you have a remote query, it has terrible distribution on the repartition streams, tons of parallel weights. Yeah.

Next time you execute it and enable it, uh, batch road heuristics. Well, that would be nice. That would be, that would be sweet. Joe mode on the. Yeah. That would be way better in my mind. Then we’re going to take a terrible row mode parallel query and maybe downgrade it.

Yeah. Cause like, cause like, if you think about a query that’s running properly, as in it’s doing a batch mode or demand based parallelism.

Yep. I can’t think of a single case where, oh, I have this great batch mode hash join. It’s running a DOP. A. Whoa, whoa, whoa. That’s terrible.

We need to knock that down to four. Like that would never, that would never happen because well, you know, just the internals are somewhat, are so much better suited to do. If you’re running a DOP, it’s too high for batch mode or demand based parallelism or broadcast parallelism.

It doesn’t really matter unless you’re running out of worker threads. But again, if you’re in batch, when you’re not going to run out of worker threads anyway. Oh, all right, Sean. Eric and I figured it out. Yep. We’ve. Ship it.

You’ve got ship it. Nothing else to do. It’s all done. Yep. You guys did it. We did. We did. Congratulations. Thank you. You should, we should get hired as co-PMs to just fix SQL Server. Cop them.

Yep. Anything else on IQP to make it IIQP in the future? Hmm. Nah. I’ve talked about everything I want to.

Yes. Wait, wait, did you want to complain about the SPX QSQL? I mean, my, my complaint with that is I don’t understand why you would want a similarly bad behavior to occur.

Compile storms. Yeah. So like I, I’ve had a number of clients where they’ve had store procedures run and all of a sudden, like one of the queries is compiling a plan and you have compile locks and now you have a bunch of other copies of the store procedure waiting to compile a plan and are waiting to use the plan that the current store procedure is compiling.

And I fixed that at quite a few places by using SPX QSQL so that now you just have like a bunch of queries compiling independently. None of them really sit around waiting.

Everyone, everyone’s happy. Like they don’t have like one store procedure that just locks up a million other copies of the store procedure. And now that’s all going to get broken, right? Let’s screw up my whole system.

I had a very good system. I had a very good thing worked out and now I’m getting screwed on it. Now you get more hours. I have a question on that actually, because I’m, I’ll admit I’m not familiar with compile storms or so-called compile storms.

So is the way it works. You could have many procedures. There’s no compiled plan. The first one to start compiling gets some type of lock resource. The other one’s a wait.

Yep. The first one to finish is compiling. There’s now a compile plan. The other sessions stop waiting and they use the compile plan from the first one. Correct.

So in what scenarios would that be bad? The one I’m thinking of is maybe the CPU you’re compiling on is like super busy and the first compile takes a long time.

So it holds a lock for a long time. Is that an example? It can be a lot. Yeah, there’s, there’s, there’s two main examples that I’ve dealt with. One of them is, I don’t even know if this is the word at this point. One of them is mitigatable by async stats updates.

Cause like some, sometimes the long compile lock is because of a long synchronous stats update. But then other, you’ll run into other times where it’s a long, like just like a long query compilation.

And like, for whatever reason that just sucks the life out of the server. Like I said, Joe, as a real life example, there’s a store procedure that was tens of tens of thousands of lines long, for example, you know, and, uh, it would take minutes sometimes to run the compilation stuff on that.

And the, I don’t know if you remember when, um, microservices was super hot. Everything was a microservice. Still kind of is.

Still kind of is. Yeah. We just don’t, it’s not super hot to say anymore. No. So, uh, we’ve moved on. Yeah. Yeah. CIOs don’t get jacked off on all over the place anymore. So, uh, anyway, so you would have this and then the, you know, microservice, oh, we need to spin up another microservice and it needs to, the query didn’t execute.

So we need to rerun it again. Check it in one second. All right. All right. Now submit it a hundred more times. And all of a sudden you have a 32, 64 core server that has 15,000 new connections getting to it.

All trying to run that query because it didn’t run the first time because microservices and you don’t care. Why are we just firing forget and hope everything goes? Well, I don’t understand that example because if it’s taking multiple minutes to compile, like wouldn’t it be worse to have many sessions all doing the multiple minute compile?

Cause, cause I feel like you’re complaining that, you know, the code’s written very poorly. Well, you need to. Right.

But you do have to pick, pick one or the other. Like if you had the compile storm, you have a bunch of sessions waiting on the lock. If you don’t pick it, you have a bunch of sessions all using CPU and other resources, trying to compile the plan.

Like they’re both bad, aren’t they? Or, or like, I don’t really see why having a bunch of sessions waiting on a lock is worse than having a bunch of sessions all doing the multiple minute compile at once.

I mean, so for that example, a, a better thing would be, let’s say it only takes 15 seconds to compile. But a lot of these, when I’ve, when I’ve worked on these, it’s been, and I mean, thousands of connections all coming in, all running the same query.

And if you would let them go and all run in the 15 and compile and do this stuff, then you’ll start getting out there. Like it’ll, it’ll, it’ll start making headway.

But if you let them all block up, then they continually block up. I mean, you’re still going to hit a point where you DDoS yourself, but that’s, there’s really not, there’s really not a good way other than not having any, even if you take out the compilation storm, it’s still not good.

Right. Yeah. All right. So I think you’re saying there’s some internals reasons as to why getting those hundreds or thousands of pile locks at the same time is going to gum things up.

Like not to, not to blame you guys in this case, cause it sounds like a pretty annoying problem, but like there’s something about the lock manager or whatever’s, or whatever’s even doing it, which makes it not able to efficiently untangle all those thousands of lot of compile locks.

Well, just think about if you have 20,000 connections, all sleeping, waiting on one lock, and then you, the lock gets freed up and then you go have to say, all right, let’s walk through 20,000 sessions and say that these are all, you know, runnable.

It just, the fact of doing that makes it ridiculous, right? You’re doing more overhead to do that than to just say, actually sit there. I’ll pick one of you.

What, none of you will get to run if we do it the other way, at least this way, one of you gets to run. Yeah. Yeah. My, my experience has been converting like, like, especially so like, like, like maybe, maybe my example was a bit oversimplified where like, there’s one query in a store procedure that like locks things up for whatever reason.

But like, you know, there can, like when you have incredibly long store procedures with a lot of, you know, like if branch logic, stuff like that, where, you know, rather than compiling a plan for everything along the line in that one go, putting this, putting stuff into the dynamic SQL gives you some separation of duties or like putting stuff into other store procedures, gives you, gives you some separation of duties, some like, some like deferred compilation stuff.

And a lot of the times that does solve a pretty big problem for the compilation locking stuff. So, you know, there’s, there’s like, I’ve just dealt with so many weird permutations of it that like the dynamic SQL thing is a very obvious answer to me.

But like, you know, like I just really hate that now dynamic SQL is going to have like that same sort of compile locking behavior. Because that, that, that really, that really undoes like a, like a behavior that I’ve relied upon for, I think the first time I ever actually had to do that was like early 2020.

So like for me, for me, that feature is a big lose because I like it, to me, it undoes a very like, like, like reliable and dependable behavior that fixed a lot of problems. Sounds like we’re all looking forward to UQP in the future, right?

Yes. New Q, New QP. U QP.

U as in intelligent. Yes. About that 15 minutes. Yep. All right. Well, I’m a good in this topic. Yep.

Me too. All right. We’re going to end this one here and we’re going to talk about standard edition next. I think if anyone can still talk after all this. Yep. Yep. I’m doing it.

I’m doing it to all of you. All right. So we’re going to stop this recording and thank you all for watching. And I hope that you’ve, you’ve found a way to survive this verbal intercourse. All right.

All right.

Going Further


If this is the kind of SQL Server stuff you love learning about, you’ll love my training. I’m offering a 25% discount to my blog readers if you click from here. I’m also available for consulting if you just don’t have time for that, and need to solve database performance problems quickly. You can also get a quick, low cost health check with no phone time required.

Fun Query Plan Friday

Fun Query Plan Friday



 

Video Summary

In this video, I dive into an absolutely fascinating query plan that Erik Darling from Darling Data explores on a Friday afternoon, perfect for winding down and enjoying some SQL Server shenanigans. We delve deep into the intricacies of a segment top operator within a cross-apply, revealing how it handles ties in user IDs and dates, which is both amusing and enlightening. The query plan itself is a delightful mix of expected and unexpected elements—like the batch mode hash join running alongside row mode operations, leading to some very peculiar bitmap handling that challenges our understanding of SQL Server’s execution plans. It’s a must-watch for anyone interested in the nitty-gritty details of how SQL Server processes complex queries, especially those involving multiple aggregations and joins.

Full Transcript

Erik Darling here with Darling Data. And today we are going to have an astoundingly obscene amount of fun, looking at what I think is a fun query plan. And since I have all the power here and I can do whatever I want, we’re going to be publishing this on a Friday. So everyone has a fun Friday and gets off to the start of a great weekend. Down in the video description, all sorts of useful links for you and me. You, so you can have SQL Server help or training, and me, so I can get money. It’s a wonderful trade-off. You know, there’s all sorts of labor theories involved here. If you want to hire me for consulting, buy my training, or support this YouTube channel, you are, I mean, I would say you are free to do all those things. You are free to do all those things, but none of those things are free. If you want to ask me office hours questions, the price of office hours questions is going to go up to $10,000 a question at the stroke of midnight, 2026. So get the free ones in while you can. And then, of course, if you like this channel and all the shenanigans therein, please do like, subscribe, tell a friend, all that good stuff.

After a long winter of solitude, I will be springing to life first at the Data Tune conference in Nashville, and that’s March 6th and 7th. And then Data Saturday Chicago, March 13th and 14th. Tickets are on sale for my Advanced T-SQL pre-cons at both of those. So if you’re in the area or just planning on attending, you don’t even have to actually attend. You can just buy a ticket and then have yourself a little Ferris Bueller day, and I won’t know the difference.

Just how many people showed up. I don’t know how many people bought tickets. I guess there’s an extra lunch. I don’t know. Now everyone gets an extra taco. Donate a lunch. You did a good deed. Look at you. What a sweetheart you are. Anyway, it is December. We have all our arms. We’ve got a ghost. Let’s go look at what I think is an interesting query plan.

So there’s an old Paul White blog post about the segment top query plan. And I want to explain a little bit about that before I get into the other parts of why this query plan is interesting. So we’ve got this sort of thing inside the cross-apply here, right?

This whole thing is inside a cross-apply. And we are selecting everything from the badges table where the date column in the badges table equals the max date column for each user, right? So we might expect to see the badges table referenced twice in the query plan because we have to get the max from the badges table here.

And then when we do all this stuff, we have to get the max per user ID, right? Because we’re correlating this here. So we might expect to see a join in this query plan, but whoa, we do not, right?

We have a scan of the one scan of the badges table. There is not a hidden scan anywhere else. I promise you there is nothing hidden anywhere else in here, which is great.

And then we have this segment operator, which I’ll show you the details of that in a moment. And then we have a top operator. So this is all fun stuff.

Now, if we look at the segment operator, we’ll see the segment operator is grouping by user ID. So this is getting the max. This is sort of getting the max date per user ID. We don’t have that aggregation in here just yet, but that’s what the top is for.

And the top is a very interesting top because the top is running as top with ties, right? And there’s a good reason for that, right? User ID and date.

And I’m going to show you the good reason for that now. Now, we have to digress from this query for a moment so I can show you. So if we were to run this query, this is… So, like, the thing whenever you’re finding…

Looking for data like this, what you might expect to see is there is some chance, because user ID is not unique and date is not unique, that there might be some users who have, like, a max date that ties, right?

So, like, you might have multiple badges that you got at the exact same time. And that does actually occur in the data, right? So if we run that same query, but we group by user ID and date, and I’m going to filter out down here and say only things where the count is greater than one, and we run this, you see that there are a whole bunch of users who have gotten a whole bunch of badges at the exact same time.

It’s like they were just awarded all these badges in one go. I think this is because the process in Stack Overflow that awards badges runs at some interval and looks for people deserving of badges and says, hey, you get a badge.

So some of these people got, like, a whole mess of badges all in one shot, right? Back when Stack Overflow was a popular website that people used and, like, you know, like, did stuff on, you know, you get all these badges for doing stuff.

So there are a lot of those. And even if we go a step further, right, and we look at this grouped by an additional element, now we’re also going to group by the badge name, right?

If we run all this, right, we still find that there are people who got a whole bunch of badges awarded at the same time, right? Just look at these top sort of numbers here.

Someone got four popular question badges at once. A bunch of people did. Look at this. Then a bunch of people got these revival badges.

I don’t know what the revival badges mean. I don’t know. Honestly, it’s been a long time since I cared about what any of the badges mean. But, you know, they got a whole bunch at the same time. Good for you, right?

You revived and populared a whole bunch of stuff. But let’s come back to this query because this is the one I think there’s interesting things in. And so we’re going to highlight this and run it with the whole thing this time, right? We’re sort of done with that initial middle section, the stuff going on in the cross apply.

And where this one gets interesting to me is we’ve already sort of examined this part, right? We have the index scan. We have our segment top section over here.

But where things get kind of neat to me is that over here we have a batch mode sort, right? You know, that’s maybe not the biggest deal. No, that’s not very exciting.

But over here we have a batch mode hash join, right? So we can see this hash match running in batch mode. But where things get kind of weird and funny to me, and this goes back to, I don’t know. I don’t know how far this went back, but so SQL Server has had bitmaps forever, right?

We have row mode bitmaps and we have batch mode bitmaps. And if we right click on the hash join operator, we’ll see that the hash join is a bitmap creator and that we created optimized bitmap 1025, right?

So this is all like running and created in batch mode. And you would normally expect to see a bitmap applied like way down over here. But apparently batch mode bitmaps do not agree with rowstore indexes, right?

There’s just like they just can’t sort of line up on that stuff. That’s just sort of, I think it’s just a bit of a technical limitation with things here. So because this runs in row mode, we can’t apply the batch mode bitmap over here.

If this executed in batch mode, then I think we could. But we got row mode execution on the reads. So we don’t get the batch mode bitmap pushed all the way down here.

Instead, the batch mode bitmap runs in this kind of funny row mode filter, right? So we have this filter operator over here. And that is where the bitmap gets applied, which is very strange, right?

It’s just a weird thing to think. Like, what happened to this bitmap? Like, did you get converted to row mode? Like, are you still batch mode? Can you be created in batch mode and you can only run in row mode?

Like, what is happening in here? It is a very, very strange, very silly query plan to me. So, of course, like when you see a lot of these plans, right? And you’re like maybe troubleshooting performance.

This might stick out to you as looking very strange. Because I always tell people that you should always be suspicious of filters and query plans. Because like almost any time you see one, it signifies some like non-relational thing that had to be expressed via a filter. It could be a very long, complex, almost non-sargable where clause.

It could be something where like you need to filter on a row number or filter on like a filter on a count. Like if we go back to the count query down here, we’ll see there’s a filter operator. But that makes total sense for there to be a filter operator here.

Because we don’t know how many rows grouped, right? We don’t know how many of the grouped rows will have a count of more than one. So, we have to generate that whole result and then filter out after we have like done the counting.

So, like sometimes you get a filter operator and you expect this when you would like expect it. Like things like this. But other times you get these weird, you get these filter operators.

And I always tell people to be very, very suspicious of filter operators. In this case, this filter operator is not that big a deal. I mean, granted it would be nice if, you know, any of this could run in batch mode.

Or this did run in batch mode and we could push the bitmap down further. But instead we get this row mode, batch mode bitmap thing over here. And we do a whole bunch of filtering.

You’ll see that this reduces rows from 1, 333, 900 to 10,000 rows. But coming back to what I was talking about earlier with the top that runs with the ties, right? Top with ties over here.

What I think is pretty amusing is that this being an inner join and all, right? We get 10,038 rows from this, right? Just from seeking into the user’s table.

But then we get 10,054 rows down here. So some of the people in here would have that same, like, tied max creation date per user ID thing going on. So I thought that was very interesting as well.

Because we end up with 10,054 rows from down here. Rather than eliminating any rows because we only got 10,038 from here. So I just thought this plan was adorable and interesting and had a lot of funny things going on.

And I wanted to talk about it. And I chose to talk about it with you. Because you’re the only people who would care if I tried to tell my wife and kids about this. They would just leave the room.

They might even leave the state, right? Country even. Who knows? I would never see them again. Anyway, happy Friday.

Thank you for watching. I hope you enjoyed yourselves. I hope you had… I hope you… I don’t know. What else? That’s it. Anyway. Get back to drinking, you lazy bums.

Alright. Goodbye. Goodbye.

Going Further


If this is the kind of SQL Server stuff you love learning about, you’ll love my training. I’m offering a 25% discount to my blog readers if you click from here. I’m also available for consulting if you just don’t have time for that, and need to solve database performance problems quickly. You can also get a quick, low cost health check with no phone time required.

Stored Procedure IF Branching and Deferred Compilation In SQL Server

Stored Procedure IF Branching and Deferred Compilation In SQL Server


Video Summary

In this video, I delve into the concept of deferred compilation in SQL Server, specifically focusing on how it can mitigate issues related to if branching within stored procedures. I explain that when SQL Server initially compiles a plan for a procedure containing if branches, it often does so based on initial parameter values, which can lead to unexpected behavior later when different parameters are passed. To illustrate this, I demonstrate an example using temporary tables and show how deferred compilation can help in such scenarios by deferring the actual compilation until the temp table is populated. Additionally, I provide tips for those interested in consulting services or training, including reasonable rates for my consulting work and membership options to join our channel community.

Full Transcript

Erik Darling here with Darling Data. And this video is going to be about how deferred compilation can help with some if branching issues that you might see in SQL Server. Primarily, one of the big problems that you run into with if branching is that all of the plans in your procedure will get compiled when SQL Server first compiles a plan for the store procedure, right? So if you have if branches and you’re thinking, well, the, like the compile time things that I pass in, SQL Server is never going to look at this branch because we’re not going there yet. Wrong. It compiles a plan for the entire procedure based on the initial set of compilation values. So if some of your parameters have null values assigned to them, well, you might, you might run into some very strange stuff when the, when that branch actually does execute because you will be executing it with values. So that can get very strange. So that can get very strange. But there are, there are of course exceptions to that. And I don’t mean like option recompile at the statement level or with recompile at the store procedure level. There are things in your store procedure that might make that untrue that cause deferred compilation. So we’re going to look at an example of that today. If you would like to hire me for consulting, because that’s what I do for a living. I consult and I make SQL Server.

I deliver faster in exchange for money. It’s my, it’s my gig. It’s my whole thing. And remember, my rates are reasonable. Yeah. All right. Been a while since I said that on camera, I think. Uh, if you’d like to buy my training also at a reasonable rate, you can do that. Uh, if you would like to become a channel member, you have a variety of reasonable rates to choose from. Uh, monthly plans are as low as $4. Four. Crazy, I know. Uh, if you would like to become a channel member, uh, that’s, that’s the way to do that. Uh, you can ask me office hours questions, uh, you, where that, that is free. Um, if you, it’s gonna be $25, $50, because, um, you’re spending clearly, you, you’re all spending far too much money on illicit substances. And, uh, it is degrading your ability to form coherent questions.

So, you need to start giving that money to me instead. I’m, I’m a safe keeper for, I am, I, I will get incoherent. Uh, and of course, if you like this channel content, please do like, subscribe, tell a friend, all that good stuff. Uh, you can tell a family member too, if you have a family member who you would like to torture with. These videos. Uh, advanced T-SQL training will be coming to DataTune Nashville March 6th and 7th and March 13th and 14th at Data Saturday Chicago. Uh, two wonderful events. Get out there in the world, you know, wear some sunglasses, look cool. Maybe we can get incoherent together. Who knows?

Uh, two wonderful events. Don’t know what that would look like. But, uh, for now, we must, we must muddle through December, January and February, and we must do that somehow. All right. Let’s look at this thing. So, um, when you use temporary objects in store procedures, um, for everybody, that’s going to mean temp tables. For people in certain situations, uh, certain SQL Server situations, uh, that would also mean table variables.

If you are getting the table variable deferred compilation, uh, intelligent query processing feature, uh, in your query plans. Um, only you can find that out. I can’t tell you that. You have to figure that out. But for temp tables, uh, you can see this happening. So, um, let’s actually add, uh, down here. I want to add a dbcc free proc cache.

Go just in case. So we’re going to look at this store procedure. Uh, I mean, a temp table is going to get created, but it’s not going to get used, right? So like we’re going to have two joins to a temp table, the stuff, I don’t need to put data in the temp table. It’s just to show you that the temp table, uh, will cause deferred compilation. Right.

And we’re going to have that in both of these queries, but let’s make sure this store procedure is run to not use a temp table. And let’s do a dbcc free proc caching and let’s run the store procedure using the reputation parameter first, right? So if we do this and we look at the execution plan and granted, I haven’t created any helpful indexes for this.

That’s really not a performance demo. It’s just a behavioral demo. And we look at the parameter list. We will see that, uh, over here we have, uh, compile and runtime values for, uh, reputation is 500,000, right? So, uh, the initial compilation for the store procedure was done with reputation at 500,000.

And that is also getting the actual execution plan, the runtime value for this execution. The reputation parameter gets used in this query against the users table. The score parameter gets used against the post table in this query.

So, uh, if we run this now and we say, uh, for score equals nine, nine, nine, nine, that’s four nines. And we look at the execution plan and we do the same thing as last time. We look at the parameter.

Where are you hiding from me? There we go. We look at the parameter list. We will see that the runtime value was nine, nine, nine, nine. But that’s not right.

That’s not right. The, uh, compile time value for this is null. All right. So SQL Server sniffed a null value and it’s doing cardinality estimation based on that null value. We have a one row estimate here, right?

So we got 12 rows back. There was one when we SQL Server estimated one row for null. We found 12 rows, uh, uh, for based on what we were looking for, which I don’t know, is that weird? Maybe, but, um, you know, not, not really.

There were four rows with a score greater than or equal to nine, nine, nine, nine. But, uh, there was a one row estimate from the null compile time value. So, uh, this is, you know, something worth noting.

If you are, if you have lots of if branching in your store procedures and there are lots of different parameters that get used in different if branches. And as you pass things in, you might actually supply different values up here.

Things can get pretty weird, right? It’s not, it’s like, like a, like another layer to a parameter sensitivity issues. But what I want to show you now is if we go and we say we put these left joins in, and I’m only using these very spurious left joins to show you that the behavior of involving a temp table.

This is the deferred compilation that you get, uh, from all temp tables and some table variables depending on your SQL Server version edition and, uh, all that good stuff. Uh, we rerun this and recreate the store procedure.

Let’s clear out the plan cache just to make sure. And what we’re going to do is the same old boring thing here. Well, we run this for reputation equals 500,000 first. And we’re going to get about the same thing happen for the first execution where the, um, compile and runtime values are both 500,000.

But now if we run this for score equals 9, 9, 9, 9. And we look at the, look at the execution plan. Of course, you got to flip that around a little bit.

Now in our parameter list, we have a compile and runtime value of 9, 9, 9, 9. So if you’ve heard me go on and on about if branching and store procedures and how, you know, like, like, like the behavior that we looked at without the temp tables can mess you up. Um, if you’re using temp tables in your if branches, you might not see that exact behavior because SQL Server might be deferring compilation of those, uh, queries until, uh, the temp table is populated, uh, and the query runs against them.

Right. So until SQL Server has to compile a plan that uses that temp table, then you might see this instead of the other behavior where it’s sniffed and null, like in the last set of, uh, runs of this. So just kind of something kind of interesting there.

Um, that’s about it on this one is usual. I don’t know if that there’s a very loud truck horn outside. I don’t know if that’s picking up on the microphone and it won’t know until the recording is complete, but it was perfectly timed with the silence there.

Anyway, I hope you enjoyed yourselves. I hope you’ll learn something and I will see you in tomorrow’s video. Uh, I forget what day tomorrow is.

I usually do, but we’re going to be there and we’re going to have fun. All right. Thank you for watching.

Going Further


If this is the kind of SQL Server stuff you love learning about, you’ll love my training. I’m offering a 25% discount to my blog readers if you click from here. I’m also available for consulting if you just don’t have time for that, and need to solve database performance problems quickly. You can also get a quick, low cost health check with no phone time required.

A Neat Trick with Using SELECT to Assign Variable Values

A Neat Trick with Using SELECT to Assign Variable Values


Video Summary

In this video, I delve into the nuances of variable assignment in SQL Server using `SET` and `SELECT`. Specifically, we explore how these commands behave differently when no rows are found, highlighting the peculiarities of `SET` where it overwrites variables with nulls, while `SELECT` maintains or overwrites them based on the result set. I demonstrate a clever workaround to make `SET` behave more like `SELECT`, especially useful in scenarios involving multivariable assignment within loops. This technique ensures that your code remains robust and avoids potential infinite loops or unexpected behavior when dealing with non-existent rows, providing a valuable lesson for anyone working with dynamic SQL assignments.

Full Transcript

Erik Darling here with Darling Data. And we have a fun little video today where we’re going to talk about how, like, if you, like, back in, like, the, I think if you watched, like, the, like, the earlier, like, Learn T-SQL, either, if you, if you were kind enough to purchase the course, well, thank you. But if you saw some of the, like, preview videos here on YouTube, one of, in one of them I talk about the difference between, some of the differences between set and select when assigning variable values and how the behavior is strange and how, like, set is kind of annoying because you can’t do, like, multiple variable assignments with it. But you can with select, but select and set act differently when a row is not found and how variable values are either maintained or overwritten with nulls.

So we’re going to talk about how to make select behave more like set in that video when you’re doing multivariable assignment. I mean, technically it would work with single variable assignment too, but you may not want to write this into every single query. Anyway, down in the old video description, boy, oh boy, if you, if you want to hire me for consulting, you, you can do it.

The power is in your hands. As long as the credit card is in your hands, the power is in your hands. You can buy my training.

You can become a supporting member of the channel. Likewise, you can do things for free. Free stuff.

Ask me office hours questions, though that may soon cost $5. $10. Maybe up to $25. Because some of the questions that come in, I think, I think, I think some, some form of angel dust was involved with that.

And if you enjoy this content, you can, of course, like, subscribe, and tell a friend or two or three or a thousand. However many you have, just break out the old Rolodex, flip through, shoot everyone a fax, say, hey, check out that Darling Data YouTube channel about SQL Server. Out in the world, yeehaw!

Nashville and Chicago in March. Back to back weekends, 6th and 7th, 13th and 14th. I will be doing pre-cons at both on Advanced T-SQL. And you should go.

You should come see me. You should go to the events. You should get out and support the wider data community before, you know, I don’t mean to sound morbid here, but any time you do something could be the last time you do it. So, get out into the world and do some, do some good and fun things, like come to data platform events.

Anyway, Merry Christmas. Let’s look at how to make set behave more like select. So, the first thing we have to do to set this adventure up is look at a couple selects from the users table.

You will notice that if I select, and I’m going to show off a fun SSMS 2022 thing here. I’m going to dynamically zoom in on these results. ID5 returns Mr. John Galloway.

I don’t know John personally. He seems wonderful. I appreciate when, I appreciate that he is in my first name boat where everyone probably looks at his name like in an email or something like that and still finds a way to spell it incorrectly when they type it. So, we have John Galloway here for ID5.

And then for ID6, we turn no rows, right? There is not a single result down here. There is nothing. I’m not hiding anything from you, I promise. But that is for ID6.

Now, if we were to do this, let’s say under normal circumstances, and we were to run this set of queries here, where we declare some variables, and we set them equal to stuff for ID5, and then we look at the contents, and then we set them equal to ID6, and we look at the contents. These are the results that we get back, right? Because no row is produced for ID6, nothing is overwritten, right?

We may just have the same variable values in here. Golly. Anyway.

Pardon me. We have this here. So, this can get people into a lot of trouble if they are unaware of this behavior and they are assigning variables to something in a loop. And then all of a sudden, they stop finding new values.

And so, they just keep either assigning nothing to this row and, like, reprocessing whatever row is in that current set of variables. Or, I don’t know, just, like, infinite loop, right? They’re just, like, they’ll, like, process this thing multiple times, or they’ll end up in an infinite loop processing the same thing over and over again, right?

Not a good time. Something to be very much aware of when writing this sort of thing, writing anything that does variable assignment. Now, set, of course, doesn’t do this, but you can’t do multivariable assignment with set because what happens is you quickly find out you can’t do it, right?

Unless you hit the, like, we would have to write three separate set queries to the user’s table in order to have that work. So, something that I picked up while working with a client was that you can have a dummy row, right? And you can do something like this with the values clause.

You could, of course, just say select null or something if you felt like it. You don’t have to use values, but I like to use values because I don’t have enough of them. Quite frankly, there’s a real shortage of values in the Darling household.

So, every opportunity I have to use values, gosh darn it, I try to get it in there. So, what you can do is something like this where you say select from values and then outer apply to your query that would do the variable assignment and then at some point do the variable assignment out here. You couldn’t do it in here.

That wouldn’t work. So, if we do this and actually I think we have a few things to run down here. Run all this stuff. Now, what we have is John Galloway set when we hit ID 5 and then we have overwritten those rows with null.

Or rather, we have overwritten those variables with null values when we did not find a row for ID 6. So, this is a nice way of making sure that your loop hits some null check condition or something when it doesn’t find anything to do. And you can do this without having to like figure out like, hey, what are the current, like what are the last set of values that I just processed?

Do I have that same set of values? Or like something like that. You know, I guess you could, you know, if you’re feeling fancy, you could do something with like row count and be like, well, did I find any rows there?

No? Okay. Well, this is just another way of doing that. This might actually be a little bit more flexible in a lot of ways because, you know, if you find nulls, you might have to go do something else or something.

Right? You don’t want to necessarily like break out of stuff. You just might want to say, hey, these are all null. Or, hey, one of these is null and it shouldn’t be.

Or, hey, two of these are null and they shouldn’t be. Or something like that. Right? There’s like all sorts of things that, you know, having this assigned would allow you to, you know, maybe explore within your loop and say, hey, that ain’t right. So, just a neat kind of trick that I learned about.

And I thought you would enjoy. That’s all I had for this one. Thank you for watching. I hope you enjoyed yourselves.

I hope you learned something. And I will see you in tomorrow’s video. I do forget what the topic is. You’ll have to forgive me. But it is one of these three windows over here. So, there is something more to talk about.

Isn’t that wonderful? All right. Thank you.

Going Further


If this is the kind of SQL Server stuff you love learning about, you’ll love my training. I’m offering a 25% discount to my blog readers if you click from here. I’m also available for consulting if you just don’t have time for that, and need to solve database performance problems quickly. You can also get a quick, low cost health check with no phone time required.

A New Query Hint to Override Batch Mode on Row Store Heuristics

A New Query Hint to Override Batch Mode on Row Store Heuristics


Video Summary

In this video, I share an exciting new query hint that I recently learned from a fellow data enthusiast, who prefers to remain anonymous for now. This hint allows you to override SQL Server’s batch mode heuristics, which can be particularly useful when the default settings don’t quite meet your needs. I demonstrate how this works by showing a simple example where applying the “use hint override batch mode heuristics” option changes an index scan and top sort from row mode to batch mode on rowstore, highlighting its potential benefits. Additionally, I point out some quirks in SQL Server Management Studio 2022 that might affect your experience when using this feature, such as a keyboard shortcut issue with the execution plan tab. Overall, this hint offers a handy workaround for situations where other methods of enabling batch mode on rowstore are not feasible.

Full Transcript

Erik Darling here with Darling Data. And we have an exciting video for you today. Because, not me, but someone out there in the world who is very handy with a debugger, I’m going to protect their name because I don’t necessarily want them to get yelled at by Microsoft in case this is explosive information. But there was a new query hint that I was made aware of from Russia with love. And it is something that I think I am going to find very, very useful in my life. And it is a query hint that allows you to override the batch mode heuristics. And by this I mean SQL Server in 2019 introduced the batch mode on rowstore feature.

The problem is that even with that feature, like SQL Server 2019 Compat Level 150 Enterprise Edition, even if you check enough boxes to get to the point where SQL Server will start applying heuristics to your queries, those heuristics may not always kick in when you want them to. And you may like have to find yourself doing stupid things in order to have batch mode on rowstore kick in for your queries. And you may not want that because the heuristics might be stupid. If you remember my parameter sensitivity training videos, I talked a lot about the parameter sensitive plan optimization and how it has heuristics and decides when or when not to kick in, which you may or may not disagree with.

And then furthermore, of course, we talked about its poor bucketing strategies, but that is way too much for this video. We have other cool stuff to talk about. So down in the video description, you’ll see all sorts of helpful links. You can hire me for consulting, you can buy my training and you can become a supporting member of the channel.

These are all things that do require you giving me money. You can do things for free. We all like free things.

You can ask me office hours questions. You know, I do appreciate a thoughtful question from the audience. And of course, if you enjoy this content, you can like, subscribe, tell a friend, all that good stuff.

If you want to see me out in the world, it’s going to be a few months, but hey, that’s okay. It’s going to be cold and you’re not going to, you know, you wouldn’t like me when I’m cold anyway. Data Tune in Nashville, March 6th and 7th.

Tickets are on sale for pre-cons there. And Data Saturday, Chicago. Well, tickets are on sale for pre-cons and a regular event in both of these things. And then Data Saturday, Chicago, March 13th and 14th.

I’ll be there as well with the pre-cons. So buy my book. Buy my book. Buy my book.

And so come see me out in the world when I’m nice and warm. Maybe I’ll wear some shorts. Who knows? We might get crazy. Anyway, I remembered to change my deck image because we are now in the December month. And I better watch out.

I think there’s a mistletoe on there. So there might be some smooches in your future. But it is a Christmas miracle. All of our friends have all of their arms. I think little Timmy over there grew his arm back.

So we can all… We still have a ghost in here. A leftover ghost. A Christmas ghost. Not sure if this is past, present, or future.

But we have a Christmas ghost. Leftover from Halloween. So I guess we didn’t take all the decorations down. Some of our decorations have been repurposed.

Anyway. Let’s talk about this hint. So first thing I’m going to do is just show you this wonderful hint. It is a use hint.

So we need to do the whole option thing. And we need to say use hint. Override batch mode heuristics. Rolls right off the tongue as many of these hints do. This hint will not appear in the sys.dm exec valid use hints DMV.

Much like some of our other favorite hints like enable parallel plan preference. Which, you know… Again, since Microsoft started using this in their code to create the disk and vector indexes.

I assume it’s safe for everyone to use in production. Because Lord knows they don’t test crap. But we’ve got this one now.

How nice. How lovely. So the first thing I want to do is just show you the query plan for this without the hint. Applied.

Applied. And if we run this and we look at the execution plan. It’s nothing terrible. But we can see that this index scan… This is not like, wow, look at how much better performance is. This is just to show you that it’s there and functional.

Then this is like the first demo query that I had where I could show you that quickly. And I just wanted to get this out quickly. So, you know, I guess more interesting stuff will happen in the future. But we can see the actual and estimated execution mode for this index scan is row.

Likewise, we can see this top end sort also occurring in row mode. All right. Cool.

Most of the other operators in the plan are not eligible for batch mode anyway. For example, the repartition streams. But rather the parallel exchanges. So gather streams, distribute streams, gather streams, blah, blah, blah.

Nested loops, not eligible. This sort would be eligible. But this top would not be eligible. But now… Actually, this is a good time to tell you about a strange buggy thing that currently is a problem in SSMS 22.

SQL Server Management Studio 2022. Is that control and R, the keyboard shortcut that allows me to quickly sort of hide results, doesn’t work from the execution plan tab. I’ve opened up an issue about it.

But you have to either click back to the script or click back to messages or results in order to hide them. Which is a little strange. I don’t know how or why that happened. But I’m sure Aaron and co will get that fixed very quickly.

But anyway, if we allow this hint to work its magic and do its thing. Override batch mode heuristic. Look how patriotic this is.

Look at this red, white, and blue over here. Hello, dark mode. If we run this now and we look at the execution plan, we will see this index scan now operates in batch mode on rowstore. Right?

There’s batch mode and there’s rowstore. And we will also see that our top end sort over here is also in batch mode also on rowstore. But you can see the batch happening there.

This sort I don’t think gets it. Oh, thanks tooltip. No, this sort for some reason remains batch mode free. But these two operators over here changed to batch mode. So if you are in a situation where, and again, control and R coming back to bite me.

If you’re in a situation where you want to see if batch mode on rowstore will work for you. And for some reason the other more common tricks for getting it to work are not options for you. Such as creating an empty non-clustered columnstore index filtered to a result that can produce no rows.

Or, you know, doing the left join on 1 equals 0 to a columnstore organized object that is empty. And you can’t make those changes. Then, you know, this hint might be the one for you.

I think the other nice and convenient thing about this hint is that you can create plan guides and stuff in query store. Or using it. And then you could, you know, have SQL Server use query plans and stuff that apply the hint.

So there are upsides to this that perhaps the other tricks do not make available to you. Anyway, something I learned. Something I thought I’d pass on to you.

Because I like you. I think you’re smart and you’re funny. You’re good looking. Everyone likes you. And you, I don’t know.

Let’s just roll out some superlatives here. I think you’re the most. What? Just that. Anyway, thank you for watching.

I hope you enjoyed yourselves. I hope you learned something. And I will see you in tomorrow’s video where we will talk about something equally compelling in the world of SQL Server. All right.

Goodbye.

Going Further


If this is the kind of SQL Server stuff you love learning about, you’ll love my training. I’m offering a 25% discount to my blog readers if you click from here. I’m also available for consulting if you just don’t have time for that, and need to solve database performance problems quickly. You can also get a quick, low cost health check with no phone time required.