Live SQL Server Q&A!

ICYMI


Last week’s thrilling, stunning, flawless episode of whatever-you-wanna-call-it.

Video Summary

In this video, I dive into the daily struggles of working in a hot environment that seems to never end. From the moment I step into my office, it feels like I’m stepping into an oven, and by midday, I’m drenched in sweat. Despite the discomfort, I manage to keep things light-hearted as I address various technical questions from the community. We tackle topics ranging from database performance tuning and server hardware requirements to antivirus software issues and the latest SQL Server versions. It’s a mix of practical advice and personal anecdotes that hopefully provide some relief and useful insights for anyone navigating similar challenges in their work or learning journey.

Full Transcript

It’s hot. It’s so hot. Everything is sweat. Shower five, six times a day just to try to stay ahead of me. It’s awful. Awful. It’s like, it’s so hot that I’m just like tired. I just feel exhausted from existing. I’m busy. The worst. I don’t even, I don’t know, I don’t know how to make it better. The beer comes down soon. I guess. I guess. Alright, look at this. healthy, happy crowd. Look at all these nice people. Get better air. I have great ones. I just haven’t put them in yet. That’s half the battle. I haven’t put them in yet. I have one air conditioner. It’s like 14,000 BTUs. I’m pretty sure it’s illegal in Europe to have an air conditioner like this one. It could like, probably like preserve.

meat. I have a pair of meat. If I put meat in front of it. Then I have a couple smaller ones for the smaller ones. But that big one, man. That’s something. The thing is, it’s like, I don’t know. It’s only, like only in the 70s. It’s just so human. It’s like death human. It’s like living in a mouth. And this is sitting in a tongue constantly. It’s like, constant, constant tongue in my entire being. It’s terrible. Oh, man. So, everyone excited for the weekend? Big weekend plans. Fun, interesting, exciting things to do this weekend. Yes. Yes. You do need to turn them on. Plug them in. Mount them in a window. All sorts of weird steps you have to go through before air conditioners actually work. I know. I know, man.

It’s like, I’m not going to go through the weekend. I know, man. I know. I know. I know. I know. The ADR paper came out. Looking forward to reading it. Well, you should paste the link in chat. Don’t keep it to yourself. Forest. Julie says, SoonerCon, science fiction, fantasy, literary con. Man of visitor is guest. All right, Julie. Where are you going with that? What is SoonerCon?

Weekend plans. Weekend plans. There we go. All right. Now it all makes sense. It’s like, Julie, you’re crazy. Someone call an ambulance for Julie. Wow. Darren says, Holy jeez. Darren says, Have you ever seen anyone that just maxed up on a schedule to match changing workloads during the day? No, that’s pretty creative, though. I think if most people, I mean, I want to say most people. I will say most people who would have those needs might just invest in resource governor and have different workloads run under different logins so that you can classify them and set maxed up differently.

But, you know, you can’t do that on every version of SQL Server. You can only do that on enterprise. Well, I’ve never personally seen that, though, but I guess I don’t hear that much. I don’t think that would freak me out too badly. that would have been a lot more. I don’t think that much. I don’t think that’s good.

Let’s see. Nothing interesting there. Twitter is terrible as usual. All right. Good stuff. Lee asks, Lee inquires, why does option recompile not seem to work sometimes? We had an issue the other day where it made zero difference even though we knew it was using a poor plan.

Oh, man. So there’s a lot of missing detail here. And it could just be that even with a recompile and SQL Server thought that it was choosing a good plan.

That would be my guess. You say not work, I mean, that’s a gray area for anything. But if I walk up to me and my hands are like, this doesn’t work. What’s not working?

What exactly didn’t work? I’ll give you an example of the difference. Like, I’m testing a lot of my current demos on SQL Server 2019. And I frequently complain that they don’t work anymore.

It doesn’t mean that they’re like throwing errors. It just means that they are no longer exhibiting the behavior that they used to. Screw you, SQL Server 2019.

You’re making me do work. He says, we passed the values through another set of parameters and then it used a new different plan.

Oh, so you did the optimize for unknown trick. You declared variables, set them equal to the other variables, and then did the thing. I guess that’s it.

I would like to see this dynamic sequence, too. Because I bet. Yeah.

Yeah. So when you declare local variables like that, it’s basically like saying optimize for unknown. And you can, I mean, sometimes the guesses are like okay. A lot of the times the guesses are very bad.

I probably wouldn’t want to leave that solution in place for very long. Who knows when that ticking time bomb will go off. So you know what?

I think you should post. I think you should post this on DBA.StackExchange.com. I think you should show us the dynamic SQL. I think you should show us the plans.

I think you should get us some more information. All in one place we can see. Because then you could get a great answer. From me here, you’re just going to get a very speculative answer.

The proc will be rewritten. All right, fine. Fine. I didn’t want to help anyway.

I didn’t want to help. You think about that. Unless you’re hiring me. Is that me you’re hiring me to do it? That would be awesome. Woohoo, I have a job.

I’m just kidding. I still do. On the plus side though, I have stopped applying for jobs. So I no longer feel rejected.

It turns out that I get as many jobs when I don’t apply to them as I do when I do apply to them. So I have the same number of jobs just without any threat of rejection.

So that’s nice. I feel fine. I feel a-okay.

It’s a good time. Good time. Good time. Good time. Still hot.

Still too hot. Still too hot to exist. I don’t have any plans this weekend in case you’ll want to.

I got nothing. Low key. Low key. As usual. I’m going to buy some ice cold wine.

And then just drown myself in it. That’s my big plan. There’s a lot of weird-well not weird, but there’s a lot of oddly placed school holidays right now.

I’m not saying the holidays are weird. I’m just saying the timing during the week is weird. Like today, my daughter had Tuesday and Thursday off. And so like there were weird like sleepovers and like play dates.

And I was just like, just go to school. Go to school. It’s too much work when you’re not in school.

It’s exhausting. Can’t keep track of the babies. I have a gigantic fan in here.

Actually, you can’t see it anymore. Never mind. But the gigantic fan is not helping my situation. It’s all downhill.

The opposite of Lee Brown Hill. It’s Eric Downhill. All right.

Do we have questions? Does anyone have a question? Anybody? You can make something up. Even if you know the answer, you can just make up a question. Otherwise, I’m going to go back to sweating off camera.

Which is preferable for me. Because these headphones add about 30 degrees to my head. 30 degrees.

Darren asks if I work with JSON and single server at all. No, and I probably won’t unless all of a sudden query plans get converted to JSON. Boris asks, what’s your naming convention for test machines?

My, I don’t have one. I named them SQL and then the version they are. So my test VMs are back to SQL Server 2008 with the most recent service pack. And we’ll just call it SQL 2008, 2008 R2, 2012, 2014.

Because I have no need for it. I don’t have dev and QA and test and all that. All I have is a few lowly VMs.

I just have to make sure that scripts don’t return written results on. Rex says he’s working on an issue with McAfee post upgrade or McAfee post upgrade causing 98% CPU and the application gets orphan sessions.

It sounds like someone didn’t do the right AV exclusions. And some like McAfee is trying to scan your SQL Server stuff. Like MDS, LDS, bin folder, things like that.

I’ve never really understood the point of having antivirus on a SQL Server. People aren’t generally uploading things to it. Like putting things on there.

I don’t know. That’s just me though. Seems like it causes more trouble than it’s worth. Yeah, the database server hosting the database.

Yeah, I got you. Jane says, any reason not to use SQL Server 2017 in production over SQL Server 2016? Haha, I wouldn’t use either one.

I would just wait for SQL Server 2019. Why put yourself behind the upgrade eight ball when SQL Server 2019 will be out in, I don’t know, three, four months?

I’m going to use the newest, latest, greatest, buggiest version of SQL Server that Microsoft has available in production. That sounds like a much better time than me.

I would wait for that. Let’s see. That’s a good name.

I’m going to go with that part of it. What would you recommend is essential basics to learn for a newly appointed junior DBA with no DBA or SQL experience?

Well, that depends on what you’re administering a little bit. If it’s your job to just deal with backups and check DB and uptime and looking after whatever HADR is in place and making sure that your servers have the right settings, then it’s a different story than if you’re looking at indexes and performance.

So scope your job for me. Tell me what you do day to day. I will give you a better answer. Farah says, what are common issues to watch out for on 100 core plus servers?

Oh, man. I don’t know. Not thread cool. Licensing. Licensing would be an issue that I would watch out for pretty hard.

That’s expensive. That sounds like an interesting one. I would be really careful how I set maxed out.

Really careful. Martin says he’d be worried about having enough IO to serve those cores. I would have so much RAM in that server that IO would not be an issue.

I would have just like if you can afford a 100 core server, which indicates that you have Enterprise Edition. a 100 core server that’s not a problem.

If you have a 100 core server, then there is absolutely no reason that you shouldn’t have that thing packed to the gills with memory. I don’t care if you have a 4 gigabyte database on that server. I would have like 8 terabytes of memory.

That’s an interesting point though because you’ve got to figure with, you know, per core you can do about 200 megs a second if they’re like decently fast cores. If you have a 100 core server though, I’d be a little bit curious what the clock speed is on those. If it’s over 2 I might be surprised.

Like 1.6, 1.8, somewhere in there. Those wide processors have grandpa speeds on them. They’re not fun.

I see a lot of people have to buy those too. They get a VM host and they have to get enough cores to serve a whole bunch of guests and it’s just 28, 56, 48 cores of garbage. I saw a VM server recently with AMD chips in it.

It was fast. Why did you do that to us? Why? Why?

Just cut one of your lights off. AMD in the server.

That’ll be the day. AMD’s going to start writing checks on for you. Yes, keep saying that. Monitoring.

You know what’s funny is every time I post one of these things I’m like go ahead ask your questions via email or you know what do you call it? Twitter or wherever. Like literally ask a question in any way that you want.

My DMs are open and all I get are porn bots saying hi to me. Then following me and then getting their account gets deleted shortly afterwards. I end up even lonelier than the more.

The story of my life. Porn bots and loneliness. Title of my memoirs. All right.

Waiting. Watching. Staring. All right. I’m going to give this one more shot.

No. All right. Thanks for showing up. It is too hot to stand here. Oh man.

There we go. James had to ask. James had to catfish me. James says what do you recommend the minimum hardware requirements for a production system that has a thousand gigabyte database. My current company has a setup of two cores and four gigs of RAM for all their new systems.

Are you messing with me? Like physically trying to mess with me? That’s not going to fly.

Unless there is like. Do you have users? I guess would be my question. Because if you if you don’t have if you don’t have users that will probably work really really well. Yeah.

So the the funny thing about you know where I would start with that is you know the sort of funny thing about SQL Server is that it doesn’t work with pages on disk. Anything that you want to read like presumably people want to read stuff out of that hundred gig database. And I’m willing to bet that your your tables or indexes are over the four gig mark.

So anything that SQL wants to read or do anything with it doesn’t matter if it’s insert update delete select it has to be in memory first. So.

Oh. It’s funny. It’s funny too because even if you’re on standard edition. 2012.

You can put like 96 gigs of memory in there and you can cache 64 gigs of memory right in the buffer pool. And then you can have like 16 gigs for the OS and you can have like I don’t know 30 or something 20 something in there. Just for SQL Server to use for caches and memory grants and all that other fun stuff.

And if you have SQL Server 2014 or newer 17 you can have a hundred and oh excuse you. You can have 128 gigs of RAM just for the buffer pool. You can cache that entire database in memory.

So I would go out on a limb and say that that is probably inadequate hardware for a 100 gig database. Matter of fact I would probably say that that is mostly only adequate hardware for a four gigabyte database. Maybe eight if you’re very lucky.

Maybe even 12 if you’re if you have incredibly good indexing and incredibly well written queries. But otherwise this is not going to pan out as well. So you know well I’m sure that there are you know query and index tuning opportunities available in that database.

I just don’t think that with four gigs of RAM I would I would just walk away from that server. There is absolutely no point in trying to do anything that that’s always going to suck. Yeah it’s basically like a bare minimum when that was installed.

Yeah you can hire me to do anything you want. I will do anything for money. Keep the women in my life happy.

Let’s see here. This is so Nandalusian says it’s not for me it’s for someone I’ve recently appointed so I don’t know what’s best to prioritize for learning. They got thrown in and had to learn as fast as things fell over.

I don’t want them to have the same experience. I want them to have a good rounded education being a DBA. So they’re less surprised by the challenges of the role. Yeah that makes sense.

But again you haven’t told me what they’re going to be responsible for. So if you have them being responsible for like backups and check DB then that’s the first kind of round of stuff they have to learn about. If you have them responsible for a 12 node AG they better they better get their button gear.

Wanting something about AG. You see what I mean about how the scope of the job can be different. Can we get lead to different advice.

Like if they’re just managing like a few agent jobs and maybe if they’re I don’t know just making sure that the server is up and running. That’s a that’s a totally different DBA than you know someone who’s got to go in and check out indexes and see if they make sense. All that other stuff monitor performance not just like up down red green monitoring or whatever color blind colors people have up in the world.

You know what I mean? It’s like different skill sets different things you have to do. Every city truck sounds like that.

Every single one. The city has not had a quiet truck in 25 years. Every truck sounds like an earthquake when it goes by.

Every truck. Fire the city. I don’t know other things I’m thinking about with the 100 gig database on.

And so here’s another funny thing. I’m going to go ahead and assume that that VM or that server that it’s sitting on is a virtual machine. And I’m going to tell you something very funny about virtual machines.

When you buy licensing for them they have a minimum of four cores. So you’re actually you’ve actually paid for two cores that you’re not using. And if that’s standard edition that’s four grand and if it’s enterprise edition that’s 14 grand.

So that’s actually a very stupid setup from a licensing perspective because you’re actually paying for things that you’re not using. And imagine if I was if I was like yeah come to my restaurant. It costs two grand per plate or per seat.

And you have to buy a minimum of four seats to come in. And then you only show up with two people. I get to keep four grand and not do any work.

You’re giving Microsoft money that they haven’t earned. All that all that licensing money that you are not using. It’s silly. It’s money out the door.

No such thing as a two core VM with SQL Server. Start with four. You’re paying for them. Why would you want to give Microsoft free money? Do you have any idea how much free money they have from not fixing anything?

Decades. You keep paying them. You get new versions with features that are broken, half implemented. Don’t really work.

You just keep paying and paying and paying. And then you give them free money with unused cores. It’s ridiculous. Absolutely ridiculous. Absolutely positively ridiculous.

Whoever’s rule that is, I would just, I would pull them in an alley and beat them. The only solution for that level of stupid is probably beatings. I can’t think of anything else that would cure that level of inanity.

So I was like, isn’t it like, that’s how everything starts out. And then like, if there’s a problem, they can magically find more hardware. Or is it like, if there’s a problem, it’s SQL Server’s fault.

And it couldn’t possibly be the hardware because it’s a DM or something. I hate it. Yeah.

It’s always SQL Server’s fault. Oh, Raphael don’t do that. Don’t add hot CPU. Don’t have the hot add CPU. No. It’s not good.

Yeah. Sorry to hear all that. So the first thing I would do is disable hot add CPU.

The second thing I would do is make sure that, you know, the CPU setup of the VM is not a gate. I would do like two by two by what’s that, 16, 20. Wait, you said 24 cores.

And 8, 8, 4, 16 plus 4 is 20. 8 plus 8 plus 4 is 20. I’m being told, whispering in my earpiece, it’s 20.

Cal Bay’s light. Yeah! Wow!

You have so much to do. I’m so glad that you’re here to tell me this story. so you hey you have you have much work to do my friend if you want to keep those 24 cores you’re gonna have to skew upgrade if you want to not have SQL server act like a piece of crap you should disable the hot ad CPU and then snow will probably work itself out once you work that out but yeah that’s that’s something that’s something you’re I would I would also try to take your boss’s job I don’t think he’s very good at it you can tell him I said I’m not afraid unless he’s from Yonkers I would recommend I would recommend you for your boss’s position change maxed up to 8 already I don’t even know if I would put it at 8 I might put it at 4 because that that one poor split up CPU just just might get real real in a real weird place with it at 8 I might I might even lower it for probably be my number at least until that other thing gets straight the the other thing gets straightened out I might change it to 8 man at least if it’s 4 you can like keep all your queries within a thing your boss is incompetent is the problem and I don’t think your boss is gonna go check on SQL server parallelism settings pretty sure that you could make that change do your thing and if he or she he or she I don’t want to assume anyone’s role in this this great wide world has anything to say about you can say Erik Darling told them maxed up 4 eric darling tell your tell your boss that he’s wrong you shouldn’t have that job he’s micromanaging bosses shouldn’t be setting those kind of limits on their employees they should not but bosses tech bosses who manage technical employees and who outsource the management of technology to other people should not ever micromanage little things like that ever it’s a big mistake you end up with things being technically dumb especially if you don’t listen to your technical employees well the things that they know best I am very technical with SQL Server but if you expected me to do something technical or you wanted me to make a technical decision about like an availability group happen would say what do you know what availability groups you know you tell me what you want to do with the availability you tell me why it makes sense I’ll ask you hard questions about why you want to do that and see if you thought this whole thing through but I’m not gonna say you can only set maxed up can’t set it to for only eight the only change you can make this maxed up idiot why would a boss make that kind of you about boss who presumably knows nothing about SQL that’s why they hired you I don’t get fire your boss throw him on a window anyway season my physical hosts are two newman nodes with six cores each 12 cores would be the maximum recommended maximum recommended for what like to give the VM you have physical VM hosts with six cores for 12 cores what we gotta talk we have to have a long talk yeah so here’s the problem with that you have a VM host with two sockets and six cores per socket so you have 12 cores total for a VM host and you want to give 12 cores to SQL Server the issue becomes if you have anything else on that box and you have SQL Server trying to co-schedule something where it’s already co-scheduling SQL Server is already co-scheduling quite a bit even if it’s just even if it’s by itself on a VM host SQL Server is being scheduled by vSphere zero server is being scheduled by Windows and then SQL Server is being internally scheduled by itself so you have three layers of scheduling going on already and now you want to know if you throw other VMs on that host they’re also being co-scheduled on the same process is it there is co co co like recursive infinite recursion of of of co-processing going on and that’s just a really terrible idea I’m going to give you a quick example that’s a little bit more extreme than what you’re telling me about and it was I was working with a client when I was still still with Brent and these people had a four core VM I think I’ve told this at a four core VM and they were they had performance problems I said yeah I agree you do you have performance problems because you have a four core VM and you have like a 200 and something gig database you have 16 gigs of RAM and it was really really not pretty and then like the further the further we dug in it turned out that not only did the VM have four cores but it was another consultant who had set up this VM host and the the VM host itself had four cores and the VM host itself was hosting 12 other SQL Server work so there were times we will I was like something’s weird here I didn’t know that stuff until like a little bit later but when I was first trying to poke around itself you’d like run a query do it finish in simply run the same query again take four seconds on the same query again finish it’s run the same query again 11 seconds on the same query instant run the same query 25 it was like every time we ran this query it was like all over the place with how long it would take and wouldn’t you know eventually it came to light that there was no no there was no remote no one it was one no my node with four cores there were 12 other SQL servers sharing the same four cores that was it every every every of the 12 servers on that VM host were using the same four cores and everything was attempting to be scheduled at once and I think the entire box had like maybe 64 gigs of RAM it was like shittier than my laptop I’m sorry crappier than my laptop oh yeah it was a bad time with VM with VM hosts like that if you have a one-to-one mapping of your of your your your your your your guest to your host you have done yourself a tremendous disservice there’s not something you should be doing anyway whoo we have we’ve gone to almost 40 minutes which is which is almost 10 minutes longer than I usually go for anyway I’m gonna get going and try and go take my fourth shower of the day and do that because people are starting to leave now I’ve talked too long anyway goodbye have a great weekend I will see you next time hopefully hopefully hopefully everyone can make it my weekend is not installing air conditioning my weekend will be installing air conditioning and then lying in front of it making sacrifices to the god of gods of Freon that’s mine goodbye

Video Summary

In this video, I dive into the daily struggles of working in a hot environment that seems to never end. From the moment I step into my office, it feels like I’m stepping into an oven, and by midday, I’m drenched in sweat. Despite the discomfort, I manage to keep things light-hearted as I address various technical questions from the community. We tackle topics ranging from database performance tuning and server hardware requirements to antivirus software issues and the latest SQL Server versions. It’s a mix of practical advice and personal anecdotes that hopefully provide some relief and useful insights for anyone navigating similar challenges in their work or learning journey.

Full Transcript

It’s hot. It’s so hot. Everything is sweat. Shower five, six times a day just to try to stay ahead of me. It’s awful. Awful. It’s like, it’s so hot that I’m just like tired. I just feel exhausted from existing. I’m busy. The worst. I don’t even, I don’t know, I don’t know how to make it better. The beer comes down soon. I guess. I guess. Alright, look at this. healthy, happy crowd. Look at all these nice people. Get better air. I have great ones. I just haven’t put them in yet. That’s half the battle. I haven’t put them in yet. I have one air conditioner. It’s like 14,000 BTUs. I’m pretty sure it’s illegal in Europe to have an air conditioner like this one. It could like, probably like preserve.

meat. I have a pair of meat. If I put meat in front of it. Then I have a couple smaller ones for the smaller ones. But that big one, man. That’s something. The thing is, it’s like, I don’t know. It’s only, like only in the 70s. It’s just so human. It’s like death human. It’s like living in a mouth. And this is sitting in a tongue constantly. It’s like, constant, constant tongue in my entire being. It’s terrible. Oh, man. So, everyone excited for the weekend? Big weekend plans. Fun, interesting, exciting things to do this weekend. Yes. Yes. You do need to turn them on. Plug them in. Mount them in a window. All sorts of weird steps you have to go through before air conditioners actually work. I know. I know, man.

It’s like, I’m not going to go through the weekend. I know, man. I know. I know. I know. I know. The ADR paper came out. Looking forward to reading it. Well, you should paste the link in chat. Don’t keep it to yourself. Forest. Julie says, SoonerCon, science fiction, fantasy, literary con. Man of visitor is guest. All right, Julie. Where are you going with that? What is SoonerCon?

Weekend plans. Weekend plans. There we go. All right. Now it all makes sense. It’s like, Julie, you’re crazy. Someone call an ambulance for Julie. Wow. Darren says, Holy jeez. Darren says, Have you ever seen anyone that just maxed up on a schedule to match changing workloads during the day? No, that’s pretty creative, though. I think if most people, I mean, I want to say most people. I will say most people who would have those needs might just invest in resource governor and have different workloads run under different logins so that you can classify them and set maxed up differently.

But, you know, you can’t do that on every version of SQL Server. You can only do that on enterprise. Well, I’ve never personally seen that, though, but I guess I don’t hear that much. I don’t think that would freak me out too badly. that would have been a lot more. I don’t think that much. I don’t think that’s good.

Let’s see. Nothing interesting there. Twitter is terrible as usual. All right. Good stuff. Lee asks, Lee inquires, why does option recompile not seem to work sometimes? We had an issue the other day where it made zero difference even though we knew it was using a poor plan.

Oh, man. So there’s a lot of missing detail here. And it could just be that even with a recompile and SQL Server thought that it was choosing a good plan.

That would be my guess. You say not work, I mean, that’s a gray area for anything. But if I walk up to me and my hands are like, this doesn’t work. What’s not working?

What exactly didn’t work? I’ll give you an example of the difference. Like, I’m testing a lot of my current demos on SQL Server 2019. And I frequently complain that they don’t work anymore.

It doesn’t mean that they’re like throwing errors. It just means that they are no longer exhibiting the behavior that they used to. Screw you, SQL Server 2019.

You’re making me do work. He says, we passed the values through another set of parameters and then it used a new different plan.

Oh, so you did the optimize for unknown trick. You declared variables, set them equal to the other variables, and then did the thing. I guess that’s it.

I would like to see this dynamic sequence, too. Because I bet. Yeah.

Yeah. So when you declare local variables like that, it’s basically like saying optimize for unknown. And you can, I mean, sometimes the guesses are like okay. A lot of the times the guesses are very bad.

I probably wouldn’t want to leave that solution in place for very long. Who knows when that ticking time bomb will go off. So you know what?

I think you should post. I think you should post this on DBA.StackExchange.com. I think you should show us the dynamic SQL. I think you should show us the plans.

I think you should get us some more information. All in one place we can see. Because then you could get a great answer. From me here, you’re just going to get a very speculative answer.

The proc will be rewritten. All right, fine. Fine. I didn’t want to help anyway.

I didn’t want to help. You think about that. Unless you’re hiring me. Is that me you’re hiring me to do it? That would be awesome. Woohoo, I have a job.

I’m just kidding. I still do. On the plus side though, I have stopped applying for jobs. So I no longer feel rejected.

It turns out that I get as many jobs when I don’t apply to them as I do when I do apply to them. So I have the same number of jobs just without any threat of rejection.

So that’s nice. I feel fine. I feel a-okay.

It’s a good time. Good time. Good time. Good time. Still hot.

Still too hot. Still too hot to exist. I don’t have any plans this weekend in case you’ll want to.

I got nothing. Low key. Low key. As usual. I’m going to buy some ice cold wine.

And then just drown myself in it. That’s my big plan. There’s a lot of weird-well not weird, but there’s a lot of oddly placed school holidays right now.

I’m not saying the holidays are weird. I’m just saying the timing during the week is weird. Like today, my daughter had Tuesday and Thursday off. And so like there were weird like sleepovers and like play dates.

And I was just like, just go to school. Go to school. It’s too much work when you’re not in school.

It’s exhausting. Can’t keep track of the babies. I have a gigantic fan in here.

Actually, you can’t see it anymore. Never mind. But the gigantic fan is not helping my situation. It’s all downhill.

The opposite of Lee Brown Hill. It’s Eric Downhill. All right.

Do we have questions? Does anyone have a question? Anybody? You can make something up. Even if you know the answer, you can just make up a question. Otherwise, I’m going to go back to sweating off camera.

Which is preferable for me. Because these headphones add about 30 degrees to my head. 30 degrees.

Darren asks if I work with JSON and single server at all. No, and I probably won’t unless all of a sudden query plans get converted to JSON. Boris asks, what’s your naming convention for test machines?

My, I don’t have one. I named them SQL and then the version they are. So my test VMs are back to SQL Server 2008 with the most recent service pack. And we’ll just call it SQL 2008, 2008 R2, 2012, 2014.

Because I have no need for it. I don’t have dev and QA and test and all that. All I have is a few lowly VMs.

I just have to make sure that scripts don’t return written results on. Rex says he’s working on an issue with McAfee post upgrade or McAfee post upgrade causing 98% CPU and the application gets orphan sessions.

It sounds like someone didn’t do the right AV exclusions. And some like McAfee is trying to scan your SQL Server stuff. Like MDS, LDS, bin folder, things like that.

I’ve never really understood the point of having antivirus on a SQL Server. People aren’t generally uploading things to it. Like putting things on there.

I don’t know. That’s just me though. Seems like it causes more trouble than it’s worth. Yeah, the database server hosting the database.

Yeah, I got you. Jane says, any reason not to use SQL Server 2017 in production over SQL Server 2016? Haha, I wouldn’t use either one.

I would just wait for SQL Server 2019. Why put yourself behind the upgrade eight ball when SQL Server 2019 will be out in, I don’t know, three, four months?

I’m going to use the newest, latest, greatest, buggiest version of SQL Server that Microsoft has available in production. That sounds like a much better time than me.

I would wait for that. Let’s see. That’s a good name.

I’m going to go with that part of it. What would you recommend is essential basics to learn for a newly appointed junior DBA with no DBA or SQL experience?

Well, that depends on what you’re administering a little bit. If it’s your job to just deal with backups and check DB and uptime and looking after whatever HADR is in place and making sure that your servers have the right settings, then it’s a different story than if you’re looking at indexes and performance.

So scope your job for me. Tell me what you do day to day. I will give you a better answer. Farah says, what are common issues to watch out for on 100 core plus servers?

Oh, man. I don’t know. Not thread cool. Licensing. Licensing would be an issue that I would watch out for pretty hard.

That’s expensive. That sounds like an interesting one. I would be really careful how I set maxed out.

Really careful. Martin says he’d be worried about having enough IO to serve those cores. I would have so much RAM in that server that IO would not be an issue.

I would have just like if you can afford a 100 core server, which indicates that you have Enterprise Edition. a 100 core server that’s not a problem.

If you have a 100 core server, then there is absolutely no reason that you shouldn’t have that thing packed to the gills with memory. I don’t care if you have a 4 gigabyte database on that server. I would have like 8 terabytes of memory.

That’s an interesting point though because you’ve got to figure with, you know, per core you can do about 200 megs a second if they’re like decently fast cores. If you have a 100 core server though, I’d be a little bit curious what the clock speed is on those. If it’s over 2 I might be surprised.

Like 1.6, 1.8, somewhere in there. Those wide processors have grandpa speeds on them. They’re not fun.

I see a lot of people have to buy those too. They get a VM host and they have to get enough cores to serve a whole bunch of guests and it’s just 28, 56, 48 cores of garbage. I saw a VM server recently with AMD chips in it.

It was fast. Why did you do that to us? Why? Why?

Just cut one of your lights off. AMD in the server.

That’ll be the day. AMD’s going to start writing checks on for you. Yes, keep saying that. Monitoring.

You know what’s funny is every time I post one of these things I’m like go ahead ask your questions via email or you know what do you call it? Twitter or wherever. Like literally ask a question in any way that you want.

My DMs are open and all I get are porn bots saying hi to me. Then following me and then getting their account gets deleted shortly afterwards. I end up even lonelier than the more.

The story of my life. Porn bots and loneliness. Title of my memoirs. All right.

Waiting. Watching. Staring. All right. I’m going to give this one more shot.

No. All right. Thanks for showing up. It is too hot to stand here. Oh man.

There we go. James had to ask. James had to catfish me. James says what do you recommend the minimum hardware requirements for a production system that has a thousand gigabyte database. My current company has a setup of two cores and four gigs of RAM for all their new systems.

Are you messing with me? Like physically trying to mess with me? That’s not going to fly.

Unless there is like. Do you have users? I guess would be my question. Because if you if you don’t have if you don’t have users that will probably work really really well. Yeah.

So the the funny thing about you know where I would start with that is you know the sort of funny thing about SQL Server is that it doesn’t work with pages on disk. Anything that you want to read like presumably people want to read stuff out of that hundred gig database. And I’m willing to bet that your your tables or indexes are over the four gig mark.

So anything that SQL wants to read or do anything with it doesn’t matter if it’s insert update delete select it has to be in memory first. So.

Oh. It’s funny. It’s funny too because even if you’re on standard edition. 2012.

You can put like 96 gigs of memory in there and you can cache 64 gigs of memory right in the buffer pool. And then you can have like 16 gigs for the OS and you can have like I don’t know 30 or something 20 something in there. Just for SQL Server to use for caches and memory grants and all that other fun stuff.

And if you have SQL Server 2014 or newer 17 you can have a hundred and oh excuse you. You can have 128 gigs of RAM just for the buffer pool. You can cache that entire database in memory.

So I would go out on a limb and say that that is probably inadequate hardware for a 100 gig database. Matter of fact I would probably say that that is mostly only adequate hardware for a four gigabyte database. Maybe eight if you’re very lucky.

Maybe even 12 if you’re if you have incredibly good indexing and incredibly well written queries. But otherwise this is not going to pan out as well. So you know well I’m sure that there are you know query and index tuning opportunities available in that database.

I just don’t think that with four gigs of RAM I would I would just walk away from that server. There is absolutely no point in trying to do anything that that’s always going to suck. Yeah it’s basically like a bare minimum when that was installed.

Yeah you can hire me to do anything you want. I will do anything for money. Keep the women in my life happy.

Let’s see here. This is so Nandalusian says it’s not for me it’s for someone I’ve recently appointed so I don’t know what’s best to prioritize for learning. They got thrown in and had to learn as fast as things fell over.

I don’t want them to have the same experience. I want them to have a good rounded education being a DBA. So they’re less surprised by the challenges of the role. Yeah that makes sense.

But again you haven’t told me what they’re going to be responsible for. So if you have them being responsible for like backups and check DB then that’s the first kind of round of stuff they have to learn about. If you have them responsible for a 12 node AG they better they better get their button gear.

Wanting something about AG. You see what I mean about how the scope of the job can be different. Can we get lead to different advice.

Like if they’re just managing like a few agent jobs and maybe if they’re I don’t know just making sure that the server is up and running. That’s a that’s a totally different DBA than you know someone who’s got to go in and check out indexes and see if they make sense. All that other stuff monitor performance not just like up down red green monitoring or whatever color blind colors people have up in the world.

You know what I mean? It’s like different skill sets different things you have to do. Every city truck sounds like that.

Every single one. The city has not had a quiet truck in 25 years. Every truck sounds like an earthquake when it goes by.

Every truck. Fire the city. I don’t know other things I’m thinking about with the 100 gig database on.

And so here’s another funny thing. I’m going to go ahead and assume that that VM or that server that it’s sitting on is a virtual machine. And I’m going to tell you something very funny about virtual machines.

When you buy licensing for them they have a minimum of four cores. So you’re actually you’ve actually paid for two cores that you’re not using. And if that’s standard edition that’s four grand and if it’s enterprise edition that’s 14 grand.

So that’s actually a very stupid setup from a licensing perspective because you’re actually paying for things that you’re not using. And imagine if I was if I was like yeah come to my restaurant. It costs two grand per plate or per seat.

And you have to buy a minimum of four seats to come in. And then you only show up with two people. I get to keep four grand and not do any work.

You’re giving Microsoft money that they haven’t earned. All that all that licensing money that you are not using. It’s silly. It’s money out the door.

No such thing as a two core VM with SQL Server. Start with four. You’re paying for them. Why would you want to give Microsoft free money? Do you have any idea how much free money they have from not fixing anything?

Decades. You keep paying them. You get new versions with features that are broken, half implemented. Don’t really work.

You just keep paying and paying and paying. And then you give them free money with unused cores. It’s ridiculous. Absolutely ridiculous. Absolutely positively ridiculous.

Whoever’s rule that is, I would just, I would pull them in an alley and beat them. The only solution for that level of stupid is probably beatings. I can’t think of anything else that would cure that level of inanity.

So I was like, isn’t it like, that’s how everything starts out. And then like, if there’s a problem, they can magically find more hardware. Or is it like, if there’s a problem, it’s SQL Server’s fault.

And it couldn’t possibly be the hardware because it’s a DM or something. I hate it. Yeah.

It’s always SQL Server’s fault. Oh, Raphael don’t do that. Don’t add hot CPU. Don’t have the hot add CPU. No. It’s not good.

Yeah. Sorry to hear all that. So the first thing I would do is disable hot add CPU.

The second thing I would do is make sure that, you know, the CPU setup of the VM is not a gate. I would do like two by two by what’s that, 16, 20. Wait, you said 24 cores.

And 8, 8, 4, 16 plus 4 is 20. 8 plus 8 plus 4 is 20. I’m being told, whispering in my earpiece, it’s 20.

Cal Bay’s light. Yeah! Wow!

You have so much to do. I’m so glad that you’re here to tell me this story. so you hey you have you have much work to do my friend if you want to keep those 24 cores you’re gonna have to skew upgrade if you want to not have SQL server act like a piece of crap you should disable the hot ad CPU and then snow will probably work itself out once you work that out but yeah that’s that’s something that’s something you’re I would I would also try to take your boss’s job I don’t think he’s very good at it you can tell him I said I’m not afraid unless he’s from Yonkers I would recommend I would recommend you for your boss’s position change maxed up to 8 already I don’t even know if I would put it at 8 I might put it at 4 because that that one poor split up CPU just just might get real real in a real weird place with it at 8 I might I might even lower it for probably be my number at least until that other thing gets straight the the other thing gets straightened out I might change it to 8 man at least if it’s 4 you can like keep all your queries within a thing your boss is incompetent is the problem and I don’t think your boss is gonna go check on SQL server parallelism settings pretty sure that you could make that change do your thing and if he or she he or she I don’t want to assume anyone’s role in this this great wide world has anything to say about you can say Erik Darling told them maxed up 4 eric darling tell your tell your boss that he’s wrong you shouldn’t have that job he’s micromanaging bosses shouldn’t be setting those kind of limits on their employees they should not but bosses tech bosses who manage technical employees and who outsource the management of technology to other people should not ever micromanage little things like that ever it’s a big mistake you end up with things being technically dumb especially if you don’t listen to your technical employees well the things that they know best I am very technical with SQL Server but if you expected me to do something technical or you wanted me to make a technical decision about like an availability group happen would say what do you know what availability groups you know you tell me what you want to do with the availability you tell me why it makes sense I’ll ask you hard questions about why you want to do that and see if you thought this whole thing through but I’m not gonna say you can only set maxed up can’t set it to for only eight the only change you can make this maxed up idiot why would a boss make that kind of you about boss who presumably knows nothing about SQL that’s why they hired you I don’t get fire your boss throw him on a window anyway season my physical hosts are two newman nodes with six cores each 12 cores would be the maximum recommended maximum recommended for what like to give the VM you have physical VM hosts with six cores for 12 cores what we gotta talk we have to have a long talk yeah so here’s the problem with that you have a VM host with two sockets and six cores per socket so you have 12 cores total for a VM host and you want to give 12 cores to SQL Server the issue becomes if you have anything else on that box and you have SQL Server trying to co-schedule something where it’s already co-scheduling SQL Server is already co-scheduling quite a bit even if it’s just even if it’s by itself on a VM host SQL Server is being scheduled by vSphere zero server is being scheduled by Windows and then SQL Server is being internally scheduled by itself so you have three layers of scheduling going on already and now you want to know if you throw other VMs on that host they’re also being co-scheduled on the same process is it there is co co co like recursive infinite recursion of of of co-processing going on and that’s just a really terrible idea I’m going to give you a quick example that’s a little bit more extreme than what you’re telling me about and it was I was working with a client when I was still still with Brent and these people had a four core VM I think I’ve told this at a four core VM and they were they had performance problems I said yeah I agree you do you have performance problems because you have a four core VM and you have like a 200 and something gig database you have 16 gigs of RAM and it was really really not pretty and then like the further the further we dug in it turned out that not only did the VM have four cores but it was another consultant who had set up this VM host and the the VM host itself had four cores and the VM host itself was hosting 12 other SQL Server work so there were times we will I was like something’s weird here I didn’t know that stuff until like a little bit later but when I was first trying to poke around itself you’d like run a query do it finish in simply run the same query again take four seconds on the same query again finish it’s run the same query again 11 seconds on the same query instant run the same query 25 it was like every time we ran this query it was like all over the place with how long it would take and wouldn’t you know eventually it came to light that there was no no there was no remote no one it was one no my node with four cores there were 12 other SQL servers sharing the same four cores that was it every every every of the 12 servers on that VM host were using the same four cores and everything was attempting to be scheduled at once and I think the entire box had like maybe 64 gigs of RAM it was like shittier than my laptop I’m sorry crappier than my laptop oh yeah it was a bad time with VM with VM hosts like that if you have a one-to-one mapping of your of your your your your your your guest to your host you have done yourself a tremendous disservice there’s not something you should be doing anyway whoo we have we’ve gone to almost 40 minutes which is which is almost 10 minutes longer than I usually go for anyway I’m gonna get going and try and go take my fourth shower of the day and do that because people are starting to leave now I’ve talked too long anyway goodbye have a great weekend I will see you next time hopefully hopefully hopefully everyone can make it my weekend is not installing air conditioning my weekend will be installing air conditioning and then lying in front of it making sacrifices to the god of gods of Freon that’s mine goodbye

Going Further


If this is the kind of SQL Server stuff you love learning about, you’ll love my training. Blog readers get 25% off the Everything Bundle — over 100 hours of performance tuning content. Need hands-on help? I offer consulting engagements from targeted investigations to ongoing retainers. Want a quick sanity check before committing to a full engagement? Schedule a call — no commitment required.

Live SQL Server Q&A!

ICYMI


Last week’s thrilling, stunning, flawless episode of whatever-you-wanna-call-it.

Video Summary

In this video, I dive into some of the challenges and quirks that come with working in SQL Server, particularly focusing on entity framework queries and parameter sniffing issues. I share my experiences dealing with an entity framework-heavy workload and discuss how custom types can sometimes cause more problems than they solve. The video also delves into a question from a DBA who is outnumbered by web developers using entity framework and struggling to balance their approach. I offer advice on supporting new developers while guiding them toward best practices, emphasizing the importance of letting people learn through experience. Additionally, I address live query plans, sharing my limited but positive experiences with them, especially in demos where they can help identify bottlenecks. The video concludes with a few miscellaneous questions and observations, including some humorous remarks about technology and personal quirks like waking up at 4 AM to watch videos.

Full Transcript

All right. So… apparently YouTube decided to change which microphone it would use for no reason. You know, why not? Why… why? why not just leave a good thing alone just mess with it right mess with it the fun thing is once you’re alive you can’t apparently switch which microphone you want to use you can hear me good wonderful apparently youtube’s definition of live is far different from my definition of live my definition of live is i can be seen and heard and poked and prodded and hugged and loved but according to youtube apparently if i’m just babbling to nothing i’m still live i don’t know why my my webcam has a microphone built in so at bare minimum you should have been able to hear extreme potato quality audio from me but now we’re we’re back here back with a good one all right so now that i’ve accomplished my mission you should accomplish your mission and ask a question any of you because i know i know that there are a lot of people out there who are varying degrees of new to sql server and you probably have really interesting fun questions and i’m willing to come come out here and and answer those questions in exchange for no upvotes whatsoever so that’s fun it’s fun i get no upvotes for this youtube thumbs don’t count they’re meaningless no one’s no one’s um uh no one’s gonna care about youtube thumbs up anyway i finally did get a question via another means over over the past week uh it’s from uh a dba who will remain anonymous from new zealand uh and they had they had a fun question and it’s it’s a question that actually strikes pretty close to home this week because uh i’ve been dealing with an entity framework heavy uh sql server workload and if i can tell you one thing about entity framework queries it’s that whatever mechanism generates those things was i mean it can really only have been conceived by satan himself there’s no other way to explain it ordering things by a row number rather than just saying order by a column doing a left self join left outer self join with a where clause that led to this weird row count spool like this the things that i’ve seen this week are baffling and what’s what’s even more fun is that because uh no one who uses entity framework has any self-control whatsoever they’ve managed to select every single column in every single table and so every memory grant is nine ten sometimes 15 gigs and only like 128 megs get used it’s like every time someone threatens me with a self-tuning database i i see entity framework queries and i’m like we’re going to europe next summer and the summer after that you can escape people will find the worst possible way to use a technology every single time i i just if you start designing your applications assuming that users are both stupid and malicious i’m a stupid user i will do dumb things and i will find weird things that happen when when people do dumb things like oh i’m kind of dumb so that makes sense but malicious users people like paul white will go out of their way to find bugs that you should have been smart enough to not not have in there in the first place so there’s that to look forward to anyway peter says custom type question i spammed at you in chat with that obvi parameter sniffing otherwise caused by local vars in a proc nope nope custom types uh don’t help they are just like and var cars or bar cars or whatever like when you make a custom type it’s quite often like you don’t want people messing with it i see this a lot in applications where um well actually in one particular application i see this quite a bit but the whole point is that you know they have very um strictly defined types for their data types and so they’ll create custom types that are like far car 11 or car 15 or in decimal 18 2 so that people when they need to like like uh operate with columns and tables that you know have have those that match up to those those uh those custom types they know exactly which one to use so no unfortunately oh boy there’s a bunch of emails coming in all right all right okay i gotta get rid of those don’t want you reading those emails in my glasses zooming in anyway uh so back to entity framework what a nightmare it is and all that other stuff uh a dba who will remain anonymous we’ll call him uh no let’s just say anonymous i like that one anonymous is okay by me i’m a dba in a small town in a small island country that is quite frequently left off maps because people forget it exists i watch your your uh your videos on saturday mornings but i don’t want to get up at 4 a.m to ask a question i understand i don’t want to get up at 4 a.m for anything yet i end up getting up at 4 a.m quite often i don’t know why no matter what time i go to bed i tend to wake up at 4 a.m and feel the urge to start working i think of something i want to do and i can’t fall back asleep anyway the question says i’m the only dba in a team working working on a team working with a team of eight web developers you sir are outnumbered in a terrible way and i i i just i don’t want to answer this question as much as i want to send you weapons i want to send you lawyers guns and money to fight off these these eight criminal elements uh but anyway he’s saying they use entity framework and store procedures so they don’t just beat the tar out of sql server with with entity framework queries there’s some some sanity involved here uh and the team is usually reasonable when it comes to changing over from entity framework to store procedures when it’s appropriate because there’s just no way to get any framework to do what you want it to do uh but there’s a new developer starting there’s a new kid in town and uh the new hire feels very strongly that as much of the database model as possible should be generated generated by entity framework he’s about to be responsible for redesigning one of our databases and i’m worried he will use this as a push for poorly designed entity framework generated database schemas and not be receptive to feedback i plan to discuss my concerns with the development team lead is this something you’d be concerned about well of course of course that’s something that i would be concerned about and it’s i would be but here’s the thing you need to let people fail you need to let people try and fail and you need to not sabotage them and you need to not root for them to fail but you need to be supportive when they do and that’s the only way that you’re going to get people over to your side which is a reasonable side that you know people that the entity framework is not a panacea for database problems entity framework is not the end-all be-all of working with the database it’s great for developers who don’t know sql and don’t want to learn sql and they just want the things that they put into code to show up in an application in a reasonable amount of time so with this new new person who’s going to redesign things you should be supportive if they want to if they want if that’s how that’s the road they want to go well sometimes the only lesson learned is through experience which is which is a shame but sometimes that’s the only way to do it i wouldn’t start off arguing with them i wouldn’t start off uh trying to dissuade them from things i would just i would let them work and make progress and if they hit road bumps be there to help them get over those bumps maybe show them a way to get over those bumps that’s not entity framework then who knows maybe maybe they’re an excellent developer maybe they are the world’s foremost entity framework developer on a small island nation that of that frequently gets forgotten from maps and maybe maybe they can do this whole thing without without you messing with them it’s an entirely plausible scenario. I mean, I know it’s not going to happen, but you know, whatever. Anyway, that’s my answer for it. I think that it’s going to be a, I mean, just don’t make it a touchy political issue. Just say it’s a technology issue. People have to be aware of the limits and the good uses and bad uses of a technology. And, you know, if they try to start square pegging stuff, just say they’re square pegging stuff. Help them out.

Help them come to a reasonable place in the world. That’s the way I’d do it, you know? Anyway. Someone else from a small island, from that same small island nation that frequently gets forgotten from maps has asked a question. How useful do you find live query plans?

Not so much. You know, I, I, I’ve never used a live query plan to troubleshoot a problem. Mostly because the people who I work with are not on a version of SQL Server that has live query plans in them. I, I would use them. The only time I’ve really used them to any great extent is locally when I’m trying to write a demo demo and I’ve wrote, I’ve written too good of a demo and it’s dragged on for like five minutes or so. And I’m thinking, well, what is this plan doing? Cause I want to see, I want to see what, what’s happening in here. And so I’ll rather like live query plans are pretty useful for just seeing like, okay, where, where are we held up here?

What, what do I need to do to make this demo a little less good? And, you know, that’s about where I stick with them. I re I really like the, the new management studio version 18 query plans because they have the, I mean, they don’t have, they don’t show you the query as it progresses.

Right. They’re not, you’re not seeing the plan. You’re not like you don’t hit F5 or whatever button combination you choose and see like the execution line doesn’t immediately pop up and start running. And you don’t see like rows as they flow through and do things. And as parts get completed, the little, little, the little line arrows turn into like solid. You don’t get all that.

You don’t, you don’t get all that. But when the, when the query is done, you do get, um, uh, you do get the final plan with mostly coherent operator times attached. Of course, if you, that varies a bit between row mode and batch mode. If you’ve watched my 10 minute video that, uh, a certain intergalactic celestial being from a small island nation that frequently gets left off maps had pointed out is well worth the 10 minutes of your time. Then you may be, maybe, maybe, maybe.

Well, no, no. Why? Uh, anyway, the thing I really hate about live query plans is how often they crash management studio, but that might not be a live, live query plan problem. That might be a management studio problem. Sometimes I’m like you hit it. You get the query running and then you hit cancel and you’d like, can’t do anything to that window.

If you try to close the window, management studio crashes. If you let it go, it’ll go on forever. And like you’ll wait until you restart. It’s a nightmare. Not, not my favorite way to spend time interacting with SQL Server management studio. God’s favorite 32 bit application. I hear, I hear management studio was God’s 30 favorite 32 bit application. Marcy asks, are you wearing Swannies? Marcy, I’m going to level with you.

I don’t know what a Swannies is. Enlighten me. What is a Swannies? Darren Scott says those scope identity in the where clause were from EF as well. Good heavens. One of my favorite things that I see in some applications is where, um, when you, uh, when certain people, so like if you have an application that needs to deal with people who access things from multiple sites, right? Like say you have sites A, B, and C. And you will have, uh, a column in the table that denotes site A, B, and C. And when different users connect in, they generate context information so that the application knows which site to point them to. And so none of the tables are directly referenced in the queries. Every single table is wrapped in a view with where column equals context info, the little context info function. So user, every single user before they touch it, like they have to get routed sort of via this function to which part of the table they should be looking at. It’s a very weird way to see things. Very strange. Anyway, uh, Paul something or other says people wouldn’t use EF if it weren’t for good stuff. Yeah. Would they? Yeah. I know a fair amount of people who use heroin. I mean, I know that’s an extreme example. I know a fair amount of people who like country music. That’s an even more extreme example. You know, a fair amount of people who wear flip-flops to walk around New York City.

Jandals, if you will, if you haven’t been wiped off the map yet. I know a fair amount of people who have made very, very dumb decisions in life. I have hand and neck tattoo. That was also a weird idea, weird choice. Anyway, what can you do? What can you do at this late stage? I can only go further. I can only wait for my mother to die so I can get face tattoos.

Otherwise she’d kill me. Let’s see. Forest, Forest of McDaniel says, when do you find CPU stats from query stats DMVs the least trustworthy? Before a query has finished. Soon nothing is there. It’s lying to you. I don’t know. I’ve never, I mean, I take, I take it all with a grain of salt. You know, I know when I write blog posts and when I write training material and, you know, talk about SQL Server generally, it’s, you know, you have to correlate things a little bit.

Right? Like you don’t want to just point to one thing and say, this is the God’s honest truth about what happened. You want to look at, like, I always want to look at a few different, this is why I’m such a big fan of new query plans, getting the operator times and the CPU time and all the other kind of cool lightweight profiling stuff that we’re getting in newer versions of SQL Server and management studio.

Because it gives you multiple sources of truth, right? You can multiple sources to check, to verify, to make sure that the thing you’re seeing holds up. All right? So there’s lots of different places that you can, you can look to validate things.

You know, it’s like, it, like, there’s no reason right now. There’s no, like, unless you, like, pull the crap out of that query profile DMV that does the live query plan thing, you can’t really validate, like, the per operator stuff too easily. You can validate, like, the overall plan time stuff pretty easily. But that’s, that’s, that’s about it.

But I, I really, I really like generally the, the amount of new feedback that we’re getting, even if it’s not all down to the microsecond or whatever new unit of time Microsoft has invented to measure things to confuse us. I hear that they’re going to invent satya seconds, which are sub nanoseconds that are only, they’re only, they’re only reliable to the closest microsecond.

That’s that. I don’t know. I think everything’s unreliable. Look, even YouTube’s unreliable. YouTube screwed me here.

Change my microphone over at the beginning of a webcast. Why would you do that? Why would you do that? I’d say audio settings have changed. Why? Dummies. Marcy says, Swanwick blue light blocking glasses. No, no. These are, these are Ray-Bans. They’re a little bit old. I should probably get a new pair of glasses soon. I just particularly like these.

I’ve grown used to them. I’ve grown fond to them on, on, on being on my face. Unlike this beard. This beard’s gone. No more of that. Max Vernon says, the tats lend a bit of a cachet. I bet people tend to listen to you in person.

I don’t know. It’s, it’s, it’s funny. Like when I was younger and I hung out with people who had tattoos, having more tattoos gave you like sort of weird social status. Right. If you had, like, if you have more tattoos on someone, you, you can be like, yeah, when are you going to get that tattoo?

Like, if I, like, I was able to do my neck tattoo, be like, well, when are you going to get your neck tattoo? Wimp, you know, like make point, like, I don’t see you with hand tattoos. What’s up with that? Right. Like, there’s like some of that, like now that I hang out, a bunch of people who like, you know, tuck their t-shirt into their underwear. The, the social, social ladder is all flipped around. I’m like no one in some of these conversations, you know, you get, you get a bunch of people sitting around a table and they want to talk about, you know, something other than the, the, the small chunk of query tuning that I enjoy.

No one’s listening to me. No one wants to hear me talk about containers. No one wants to hear me talk about availability groups. No one wants to hear me talk about taking backups. I don’t want to hear me talk about taking backups. You know how dull that is?

Ugh. Nothing to do with it. Let’s check TB scheduling. Get out of here. Talk to someone who cares. You know, the only time I care about that because.

Oh, screw it. There’s a, there’s another question here. What do you think about implicit conversions and plan warnings more generally? Well, golly and gosh. Eating a lot of dinner because of implicit conversions.

Can’t complain too much about those. A lot of dinners. A lot of dinners. A lot of wine because of implicit conversions. And I’m pretty sure you’ve gotten some wine because of implicit conversions. My friend. What do I think? What do I think? I mean, if, if someone, if, if you have people out in the world who can’t be bothered to, to match data types, like what, what, what, what help is there for them?

They like this pick something randomly and compare them to random things and compare them. Like, I want to know if this knife matches his pen. What do you tell those people? What can you tell them?

I want to know if this fork is a notebook. What do you tell them? There’s no sense. There’s no, if, if people are, are that dead set on not caring, I will happily take their money to tell them they should care. That’s it.

Plan warnings more generally. We get, we get warned about the wrong things. We get warnings. We get, first off, the, the, the, the implicit conversion warnings are sometimes full of dookie. Right?

Like if, like one example that I show when I talk about them is like, if you have an integer column and you say, like, select, I let’s, let’s call it ID. Select ID as VARCAR 10. You will get an implicit conversion warning saying that the cardinality estimate of your query is incorrect.

You’ll also get an implicit conversion warnings for seek plan when there’s no index that you could possibly seek into. Right? Like let’s just say that I have, for example, another column called reputation.

And I say, select star from users where reputation equals a variable that’s, and then VARCAR something. Right? And I don’t have an index on reputation. SQL Server will say, well, you could have done a seek plan, dummy. And you couldn’t because there was no index on it.

There was nothing for you to seek into. Nothing there. Another pet peeve is the no join predicate warning, which is that, would you get that big red, can you imagine the things that Microsoft should put a big red X over in query plans? Like every spool?

Like every single spool should just have a giant red X over it. But no, we get, we get warned when completely reasonably written queries with a join predicate. I say you don’t have a join predicate. Okay.

You win. And those memory grant warnings, which may affect the reliability of me to stay sober. Right? He’s like, you’re, you’re, you had a thousand and 24 KB memory grant and you only used 176 KB of it. May affect the reliability.

I love that. I love that sentence. You know what? If there’s one fragmentation that I worry about in SQL Server, it is, it is sentence fragmentation. And these warnings affect, it may impact the reliability. It’s just like, okay, tell me more.

Tell me more. I would love to hear more. Oh God. Where are some other ones? Where are some other funny ones? Oh boy. I don’t know.

I think spills are overrated. I think spills, I think, I don’t think you should get a spill warning unless you are like really spilling. You should. Yeah.

Unmatched indexes. That was a fun one. It was, it was fun to find out that if you have a plan that’s auto parameterized, you’ll get an unmatched index warning. When, even though, even if you use the index, like you can see the index being used in the plan. But if your plan is auto parameterized or simple parameterized, whatever you want to call it, you get a warning.

We couldn’t match that index to anything. But I see you. I see you. It’s aggravating.

Aggravating. It’s like, if you could, if you could go back and start over with like the query plan, XML. Hey, would you even use XML?

Use JSON. I would, I would much rather just get like a, like an MS paint. It’s like, give me a PNG of the plans. Don’t make me use the XML. It’s like, write everything down for me. Write everything down.

Marion asks if I think Paul is real. Well, he better be. Or else, someone else has my old laptop. As far as I know, he is. Yes, plans rendered by entity framework. That would be, that would be ideal.

Because then you would never get a query plan. And you would never be able to fix anything. And you would be able to happily learn how to do something else for a living. Go learn, go learn C sharp. Go learn, go write, write a great novel.

Go write the next great novel for your country. Where is it? Next great American novel, Canadian novel. New Zealand. I mean, let’s, no one in Australia is writing a novel. Get eaten by dingoes anyway.

Dingo ate my novel. I don’t know. I often think about what I would, things that I would much rather be doing than SQL Server. The list keeps growing long.

Fun. Marcy says, speaking of weird things and plans, is there a way to keep a detail pop-up open if I want to move my cursor to another screen? No.

No. Yes.

Of plan connecting lines and all that number of rose red malarkey. Yeah, that’s, you know. What bothers me is something that I learned from you about cash plans where they are just stitched together from many different plans. So, like, when you have a plan, you see, like, a plan in your plan cache or, like, an estimated plan where you have, like, thin lines and thin lines and thin lines.

And then you hit one operator and there’s a very, very thick line coming out of that operator for some reason. And the estimate goes from, like, three rows to, like, 40 billion rows. That’s just the optimizer being kooky.

Putting the plan in cache that might have been a few other plans stapled together. Dreadful. Marcy says, I don’t know how to get a screenshot of those. They disappear as soon as I click anywhere else, like on the snipping tool.

So, I use Snagit. S-N-A-G-I-T. I use Snagit. And when I have my cursor hovering over an operator so I get the tooltip, I can just hit the print screen button. And that’ll trigger the screenshotting of my screen.

And I can zoom in on the tooltip. Snagit is great for that. It’s well worth, I forget, like, 25 bucks or something. And then, like, the occasional maintenance contract so that you get updates. Well worth it.

The other upside of Snagit is that it has, like, all sorts, like, it has, like, an editor built in. So, you can do all sorts of cool stuff with things you take pictures of. You can, like, do, like, the crop where, like, you cut out, like, a certain part either horizontally or vertically. You can draw shapes.

They have the most accusatory built-in arrows. So, if you really want to call attention to something and be like, this sucks, the arrows they have are wonderful for that. Like, some of them, like, get bigger. They, like, taper out and, like, get huge arrowheads. Awesome with that.

Good accusatory arrow really makes for a good presentation. Snagit’s good. They don’t even pay me to say that. I won’t get a coupon. No coupons for me, unfortunately.

I am surprisingly, I have never been asked to be, like, a brand evangelist. Oh, yeah. That’s my story.

Anyway, we have hit the 30-minute mark. I have babbled at you long enough. It’s getting hot in here again. I haven’t, I might have noticed I haven’t complained about it being hot this week because I plugged my air conditioners in. But the air conditioners are out there. When my door is closed, this room just slowly gets hotter and hotter because I have, like, 90 inches of screens hanging around me.

Anyway, I’m going to get going. I’m going to go get back to work. Hopefully make some money. And I will see you next week. Thank you for coming. I hope you enjoyed it.

Thank you for the great questions, and I will see you next time.

Video Summary

In this video, I dive into some of the challenges and quirks that come with working in SQL Server, particularly focusing on entity framework queries and parameter sniffing issues. I share my experiences dealing with an entity framework-heavy workload and discuss how custom types can sometimes cause more problems than they solve. The video also delves into a question from a DBA who is outnumbered by web developers using entity framework and struggling to balance their approach. I offer advice on supporting new developers while guiding them toward best practices, emphasizing the importance of letting people learn through experience. Additionally, I address live query plans, sharing my limited but positive experiences with them, especially in demos where they can help identify bottlenecks. The video concludes with a few miscellaneous questions and observations, including some humorous remarks about technology and personal quirks like waking up at 4 AM to watch videos.

Full Transcript

All right. So… apparently YouTube decided to change which microphone it would use for no reason. You know, why not? Why… why? why not just leave a good thing alone just mess with it right mess with it the fun thing is once you’re alive you can’t apparently switch which microphone you want to use you can hear me good wonderful apparently youtube’s definition of live is far different from my definition of live my definition of live is i can be seen and heard and poked and prodded and hugged and loved but according to youtube apparently if i’m just babbling to nothing i’m still live i don’t know why my my webcam has a microphone built in so at bare minimum you should have been able to hear extreme potato quality audio from me but now we’re we’re back here back with a good one all right so now that i’ve accomplished my mission you should accomplish your mission and ask a question any of you because i know i know that there are a lot of people out there who are varying degrees of new to sql server and you probably have really interesting fun questions and i’m willing to come come out here and and answer those questions in exchange for no upvotes whatsoever so that’s fun it’s fun i get no upvotes for this youtube thumbs don’t count they’re meaningless no one’s no one’s um uh no one’s gonna care about youtube thumbs up anyway i finally did get a question via another means over over the past week uh it’s from uh a dba who will remain anonymous from new zealand uh and they had they had a fun question and it’s it’s a question that actually strikes pretty close to home this week because uh i’ve been dealing with an entity framework heavy uh sql server workload and if i can tell you one thing about entity framework queries it’s that whatever mechanism generates those things was i mean it can really only have been conceived by satan himself there’s no other way to explain it ordering things by a row number rather than just saying order by a column doing a left self join left outer self join with a where clause that led to this weird row count spool like this the things that i’ve seen this week are baffling and what’s what’s even more fun is that because uh no one who uses entity framework has any self-control whatsoever they’ve managed to select every single column in every single table and so every memory grant is nine ten sometimes 15 gigs and only like 128 megs get used it’s like every time someone threatens me with a self-tuning database i i see entity framework queries and i’m like we’re going to europe next summer and the summer after that you can escape people will find the worst possible way to use a technology every single time i i just if you start designing your applications assuming that users are both stupid and malicious i’m a stupid user i will do dumb things and i will find weird things that happen when when people do dumb things like oh i’m kind of dumb so that makes sense but malicious users people like paul white will go out of their way to find bugs that you should have been smart enough to not not have in there in the first place so there’s that to look forward to anyway peter says custom type question i spammed at you in chat with that obvi parameter sniffing otherwise caused by local vars in a proc nope nope custom types uh don’t help they are just like and var cars or bar cars or whatever like when you make a custom type it’s quite often like you don’t want people messing with it i see this a lot in applications where um well actually in one particular application i see this quite a bit but the whole point is that you know they have very um strictly defined types for their data types and so they’ll create custom types that are like far car 11 or car 15 or in decimal 18 2 so that people when they need to like like uh operate with columns and tables that you know have have those that match up to those those uh those custom types they know exactly which one to use so no unfortunately oh boy there’s a bunch of emails coming in all right all right okay i gotta get rid of those don’t want you reading those emails in my glasses zooming in anyway uh so back to entity framework what a nightmare it is and all that other stuff uh a dba who will remain anonymous we’ll call him uh no let’s just say anonymous i like that one anonymous is okay by me i’m a dba in a small town in a small island country that is quite frequently left off maps because people forget it exists i watch your your uh your videos on saturday mornings but i don’t want to get up at 4 a.m to ask a question i understand i don’t want to get up at 4 a.m for anything yet i end up getting up at 4 a.m quite often i don’t know why no matter what time i go to bed i tend to wake up at 4 a.m and feel the urge to start working i think of something i want to do and i can’t fall back asleep anyway the question says i’m the only dba in a team working working on a team working with a team of eight web developers you sir are outnumbered in a terrible way and i i i just i don’t want to answer this question as much as i want to send you weapons i want to send you lawyers guns and money to fight off these these eight criminal elements uh but anyway he’s saying they use entity framework and store procedures so they don’t just beat the tar out of sql server with with entity framework queries there’s some some sanity involved here uh and the team is usually reasonable when it comes to changing over from entity framework to store procedures when it’s appropriate because there’s just no way to get any framework to do what you want it to do uh but there’s a new developer starting there’s a new kid in town and uh the new hire feels very strongly that as much of the database model as possible should be generated generated by entity framework he’s about to be responsible for redesigning one of our databases and i’m worried he will use this as a push for poorly designed entity framework generated database schemas and not be receptive to feedback i plan to discuss my concerns with the development team lead is this something you’d be concerned about well of course of course that’s something that i would be concerned about and it’s i would be but here’s the thing you need to let people fail you need to let people try and fail and you need to not sabotage them and you need to not root for them to fail but you need to be supportive when they do and that’s the only way that you’re going to get people over to your side which is a reasonable side that you know people that the entity framework is not a panacea for database problems entity framework is not the end-all be-all of working with the database it’s great for developers who don’t know sql and don’t want to learn sql and they just want the things that they put into code to show up in an application in a reasonable amount of time so with this new new person who’s going to redesign things you should be supportive if they want to if they want if that’s how that’s the road they want to go well sometimes the only lesson learned is through experience which is which is a shame but sometimes that’s the only way to do it i wouldn’t start off arguing with them i wouldn’t start off uh trying to dissuade them from things i would just i would let them work and make progress and if they hit road bumps be there to help them get over those bumps maybe show them a way to get over those bumps that’s not entity framework then who knows maybe maybe they’re an excellent developer maybe they are the world’s foremost entity framework developer on a small island nation that of that frequently gets forgotten from maps and maybe maybe they can do this whole thing without without you messing with them it’s an entirely plausible scenario. I mean, I know it’s not going to happen, but you know, whatever. Anyway, that’s my answer for it. I think that it’s going to be a, I mean, just don’t make it a touchy political issue. Just say it’s a technology issue. People have to be aware of the limits and the good uses and bad uses of a technology. And, you know, if they try to start square pegging stuff, just say they’re square pegging stuff. Help them out.

Help them come to a reasonable place in the world. That’s the way I’d do it, you know? Anyway.

Someone else from a small island, from that same small island nation that frequently gets forgotten from maps has asked a question. How useful do you find live query plans?

Not so much. You know, I, I, I’ve never used a live query plan to troubleshoot a problem. Mostly because the people who I work with are not on a version of SQL Server that has live query plans in them. I, I would use them. The only time I’ve really used them to any great extent is locally when I’m trying to write a demo demo and I’ve wrote, I’ve written too good of a demo and it’s dragged on for like five minutes or so. And I’m thinking, well, what is this plan doing? Cause I want to see, I want to see what, what’s happening in here. And so I’ll rather like live query plans are pretty useful for just seeing like, okay, where, where are we held up here?

What, what do I need to do to make this demo a little less good? And, you know, that’s about where I stick with them. I re I really like the, the new management studio version 18 query plans because they have the, I mean, they don’t have, they don’t show you the query as it progresses.

Right. They’re not, you’re not seeing the plan. You’re not like you don’t hit F5 or whatever button combination you choose and see like the execution line doesn’t immediately pop up and start running. And you don’t see like rows as they flow through and do things. And as parts get completed, the little, little, the little line arrows turn into like solid. You don’t get all that.

You don’t, you don’t get all that. But when the, when the query is done, you do get, um, uh, you do get the final plan with mostly coherent operator times attached. Of course, if you, that varies a bit between row mode and batch mode. If you’ve watched my 10 minute video that, uh, a certain intergalactic celestial being from a small island nation that frequently gets left off maps had pointed out is well worth the 10 minutes of your time. Then you may be, maybe, maybe, maybe.

Well, no, no. Why? Uh, anyway, the thing I really hate about live query plans is how often they crash management studio, but that might not be a live, live query plan problem. That might be a management studio problem. Sometimes I’m like you hit it. You get the query running and then you hit cancel and you’d like, can’t do anything to that window.

If you try to close the window, management studio crashes. If you let it go, it’ll go on forever. And like you’ll wait until you restart. It’s a nightmare. Not, not my favorite way to spend time interacting with SQL Server management studio. God’s favorite 32 bit application. I hear, I hear management studio was God’s 30 favorite 32 bit application. Marcy asks, are you wearing Swannies? Marcy, I’m going to level with you.

I don’t know what a Swannies is. Enlighten me. What is a Swannies? Darren Scott says those scope identity in the where clause were from EF as well. Good heavens. One of my favorite things that I see in some applications is where, um, when you, uh, when certain people, so like if you have an application that needs to deal with people who access things from multiple sites, right? Like say you have sites A, B, and C. And you will have, uh, a column in the table that denotes site A, B, and C. And when different users connect in, they generate context information so that the application knows which site to point them to. And so none of the tables are directly referenced in the queries. Every single table is wrapped in a view with where column equals context info, the little context info function. So user, every single user before they touch it, like they have to get routed sort of via this function to which part of the table they should be looking at. It’s a very weird way to see things. Very strange. Anyway, uh, Paul something or other says people wouldn’t use EF if it weren’t for good stuff. Yeah. Would they? Yeah. I know a fair amount of people who use heroin. I mean, I know that’s an extreme example. I know a fair amount of people who like country music. That’s an even more extreme example. You know, a fair amount of people who wear flip-flops to walk around New York City.

Jandals, if you will, if you haven’t been wiped off the map yet. I know a fair amount of people who have made very, very dumb decisions in life. I have hand and neck tattoo. That was also a weird idea, weird choice. Anyway, what can you do? What can you do at this late stage? I can only go further. I can only wait for my mother to die so I can get face tattoos.

Otherwise she’d kill me. Let’s see. Forest, Forest of McDaniel says, when do you find CPU stats from query stats DMVs the least trustworthy? Before a query has finished. Soon nothing is there. It’s lying to you. I don’t know. I’ve never, I mean, I take, I take it all with a grain of salt. You know, I know when I write blog posts and when I write training material and, you know, talk about SQL Server generally, it’s, you know, you have to correlate things a little bit.

Right? Like you don’t want to just point to one thing and say, this is the God’s honest truth about what happened. You want to look at, like, I always want to look at a few different, this is why I’m such a big fan of new query plans, getting the operator times and the CPU time and all the other kind of cool lightweight profiling stuff that we’re getting in newer versions of SQL Server and management studio.

Because it gives you multiple sources of truth, right? You can multiple sources to check, to verify, to make sure that the thing you’re seeing holds up. All right? So there’s lots of different places that you can, you can look to validate things.

You know, it’s like, it, like, there’s no reason right now. There’s no, like, unless you, like, pull the crap out of that query profile DMV that does the live query plan thing, you can’t really validate, like, the per operator stuff too easily. You can validate, like, the overall plan time stuff pretty easily. But that’s, that’s, that’s about it.

But I, I really, I really like generally the, the amount of new feedback that we’re getting, even if it’s not all down to the microsecond or whatever new unit of time Microsoft has invented to measure things to confuse us. I hear that they’re going to invent satya seconds, which are sub nanoseconds that are only, they’re only, they’re only reliable to the closest microsecond.

That’s that. I don’t know. I think everything’s unreliable. Look, even YouTube’s unreliable. YouTube screwed me here.

Change my microphone over at the beginning of a webcast. Why would you do that? Why would you do that? I’d say audio settings have changed. Why? Dummies. Marcy says, Swanwick blue light blocking glasses. No, no. These are, these are Ray-Bans.

They’re a little bit old. I should probably get a new pair of glasses soon. I just particularly like these. I’ve grown used to them. I’ve grown fond to them on, on, on being on my face. Unlike this beard. This beard’s gone. No more of that.

Max Vernon says, the tats lend a bit of a cachet. I bet people tend to listen to you in person. I don’t know. It’s, it’s, it’s funny. Like when I was younger and I hung out with people who had tattoos, having more tattoos gave you like sort of weird social status. Right.

If you had, like, if you have more tattoos on someone, you, you can be like, yeah, when are you going to get that tattoo? Like, if I, like, I was able to do my neck tattoo, be like, well, when are you going to get your neck tattoo? Wimp, you know, like make point, like, I don’t see you with hand tattoos. What’s up with that?

Right. Like, there’s like some of that, like now that I hang out, a bunch of people who like, you know, tuck their t-shirt into their underwear. The, the social, social ladder is all flipped around. I’m like no one in some of these conversations, you know, you get, you get a bunch of people sitting around a table and they want to talk about, you know, something other than the, the, the small chunk of query tuning that I enjoy. No one’s listening to me. No one wants to hear me talk about containers.

No one wants to hear me talk about availability groups. No one wants to hear me talk about taking backups. I don’t want to hear me talk about taking backups. You know how dull that is? Ugh.

Nothing to do with it. Let’s check TB scheduling. Get out of here. Talk to someone who cares. You know, the only time I care about that because.

Oh, screw it. There’s a, there’s another question here. What do you think about implicit conversions and plan warnings more generally? Well, golly and gosh. Eating a lot of dinner because of implicit conversions.

Can’t complain too much about those. A lot of dinners. A lot of dinners. A lot of wine because of implicit conversions. And I’m pretty sure you’ve gotten some wine because of implicit conversions. My friend.

What do I think? What do I think? I mean, if, if someone, if, if you have people out in the world who can’t be bothered to, to match data types, like what, what, what, what help is there for them? They like this pick something randomly and compare them to random things and compare them.

Like, I want to know if this knife matches his pen. What do you tell those people? What can you tell them? I want to know if this fork is a notebook.

What do you tell them? There’s no sense. There’s no, if, if people are, are that dead set on not caring, I will happily take their money to tell them they should care.

That’s it. Plan warnings more generally. We get, we get warned about the wrong things.

We get warnings. We get, first off, the, the, the, the implicit conversion warnings are sometimes full of dookie. Right?

Like if, like one example that I show when I talk about them is like, if you have an integer column and you say, like, select, I let’s, let’s call it ID. Select ID as VARCAR 10. You will get an implicit conversion warning saying that the cardinality estimate of your query is incorrect.

You’ll also get an implicit conversion warnings for seek plan when there’s no index that you could possibly seek into. Right? Like let’s just say that I have, for example, another column called reputation.

And I say, select star from users where reputation equals a variable that’s, and then VARCAR something. Right? And I don’t have an index on reputation.

SQL Server will say, well, you could have done a seek plan, dummy. And you couldn’t because there was no index on it. There was nothing for you to seek into. Nothing there.

Another pet peeve is the no join predicate warning, which is that, would you get that big red, can you imagine the things that Microsoft should put a big red X over in query plans? Like every spool? Like every single spool should just have a giant red X over it.

But no, we get, we get warned when completely reasonably written queries with a join predicate. I say you don’t have a join predicate. Okay.

You win. And those memory grant warnings, which may affect the reliability of me to stay sober. Right?

He’s like, you’re, you’re, you had a thousand and 24 KB memory grant and you only used 176 KB of it. May affect the reliability. I love that.

I love that sentence. You know what? If there’s one fragmentation that I worry about in SQL Server, it is, it is sentence fragmentation. And these warnings affect, it may impact the reliability.

It’s just like, okay, tell me more. Tell me more. I would love to hear more.

Oh God. Where are some other ones? Where are some other funny ones? Oh boy. I don’t know.

I think spills are overrated. I think spills, I think, I don’t think you should get a spill warning unless you are like really spilling. You should.

Yeah. Unmatched indexes. That was a fun one. It was, it was fun to find out that if you have a plan that’s auto parameterized, you’ll get an unmatched index warning. When, even though, even if you use the index, like you can see the index being used in the plan.

But if your plan is auto parameterized or simple parameterized, whatever you want to call it, you get a warning. We couldn’t match that index to anything. But I see you.

I see you. It’s aggravating. Aggravating. It’s like, if you could, if you could go back and start over with like the query plan, XML.

Hey, would you even use XML? Use JSON. I would, I would much rather just get like a, like an MS paint.

It’s like, give me a PNG of the plans. Don’t make me use the XML. It’s like, write everything down for me. Write everything down.

Marion asks if I think Paul is real. Well, he better be. Or else, someone else has my old laptop. As far as I know, he is.

Yes, plans rendered by entity framework. That would be, that would be ideal. Because then you would never get a query plan. And you would never be able to fix anything. And you would be able to happily learn how to do something else for a living.

Go learn, go learn C sharp. Go learn, go write, write a great novel. Go write the next great novel for your country.

Where is it? Next great American novel, Canadian novel. New Zealand. I mean, let’s, no one in Australia is writing a novel. Get eaten by dingoes anyway.

Dingo ate my novel. I don’t know. I often think about what I would, things that I would much rather be doing than SQL Server.

The list keeps growing long. Fun. Marcy says, speaking of weird things and plans, is there a way to keep a detail pop-up open if I want to move my cursor to another screen?

No. No. Yes.

Of plan connecting lines and all that number of rose red malarkey. Yeah, that’s, you know. What bothers me is something that I learned from you about cash plans where they are just stitched together from many different plans. So, like, when you have a plan, you see, like, a plan in your plan cache or, like, an estimated plan where you have, like, thin lines and thin lines and thin lines.

And then you hit one operator and there’s a very, very thick line coming out of that operator for some reason. And the estimate goes from, like, three rows to, like, 40 billion rows. That’s just the optimizer being kooky.

Putting the plan in cache that might have been a few other plans stapled together. Dreadful. Marcy says, I don’t know how to get a screenshot of those.

They disappear as soon as I click anywhere else, like on the snipping tool. So, I use Snagit. S-N-A-G-I-T.

I use Snagit. And when I have my cursor hovering over an operator so I get the tooltip, I can just hit the print screen button. And that’ll trigger the screenshotting of my screen.

And I can zoom in on the tooltip. Snagit is great for that. It’s well worth, I forget, like, 25 bucks or something. And then, like, the occasional maintenance contract so that you get updates.

Well worth it. The other upside of Snagit is that it has, like, all sorts, like, it has, like, an editor built in. So, you can do all sorts of cool stuff with things you take pictures of.

You can, like, do, like, the crop where, like, you cut out, like, a certain part either horizontally or vertically. You can draw shapes. They have the most accusatory built-in arrows.

So, if you really want to call attention to something and be like, this sucks, the arrows they have are wonderful for that. Like, some of them, like, get bigger. They, like, taper out and, like, get huge arrowheads.

Awesome with that. Good accusatory arrow really makes for a good presentation. Snagit’s good.

They don’t even pay me to say that. I won’t get a coupon. No coupons for me, unfortunately. I am surprisingly, I have never been asked to be, like, a brand evangelist. Oh, yeah.

That’s my story. Anyway, we have hit the 30-minute mark. I have babbled at you long enough. It’s getting hot in here again. I haven’t, I might have noticed I haven’t complained about it being hot this week because I plugged my air conditioners in. But the air conditioners are out there.

When my door is closed, this room just slowly gets hotter and hotter because I have, like, 90 inches of screens hanging around me. Anyway, I’m going to get going. I’m going to go get back to work.

Hopefully make some money. And I will see you next week. Thank you for coming. I hope you enjoyed it. Thank you for the great questions, and I will see you next time.

Going Further


If this is the kind of SQL Server stuff you love learning about, you’ll love my training. Blog readers get 25% off the Everything Bundle — over 100 hours of performance tuning content. Need hands-on help? I offer consulting engagements from targeted investigations to ongoing retainers. Want a quick sanity check before committing to a full engagement? Schedule a call — no commitment required.

Live SQL Server Q&A!

ICYMI


Last week’s thrilling, stunning, flawless episode of whatever-you-wanna-call-it.

Thanks for watching!

Video Summary

In this video, I dive into some of the exciting new features coming in SQL Server 2019, particularly focusing on batch mode memory grant feedback and adaptive joins. These features are only available if you’re using compatibility level 150, which presents a unique set of considerations for database administrators. I also delve into key lookups, explaining their role in execution plans and how to identify potentially expensive ones using the SQL Server Blitz cache procedures. The discussion then shifts to practical advice on running SSIS and Reporting Services (SSRS) services alongside the SQL Server engine, emphasizing the importance of keeping production environments as clean and efficient as possible. Throughout the video, I share some personal insights, including my ongoing journey towards achieving a long-held goal and reflecting on recent changes in my professional landscape, such as the exciting news about Zane Brunette joining Microsoft. The conversation also touches on the nature of working alone versus being part of a team, with a humorous nod to my gym routine and its impact on my current state of mind—hot, hangry, and ready for some serious database tuning challenges!

Full Transcript

It is so hot. It’s so hot and I’m so annoyed with all the heat. I finally got to eat lunch. Prior to that, I was both hot and hangry at the same time. My hair is sweaty, I’m sorry. I look like garbage. I look like garbage. It’s not fun. It’s not fun. I should probably mention that I’m doing this somewhere. I don’t think I did that. The one person who is here. Thank you for showing up. Thank you for having faith in me. One person who is here. Hopefully you’re not just some Russian bot.

Here to collude. Here to collude with me about things. Oh, and you’re gone. Goodbye, Russian bot. Goodbye forever. Do-do-do-do. There, I’ve tweeted about it now, so no one has an excuse for not showing up.

So, if no one shows up on this beautiful Friday, it’s a hot Friday, then I’m going to talk about something that’s been on my mind lately. And that something is the coming… Off to a good start. I’m not going to say the word coming correctly. The coming reckoning. That’s where a coming came from, was coming reckoning.

The coming reckoning that is due with SQL Server 2019. So, there were things introduced in SQL Server 2017 that I thought were really cool. And that was batch mode memory grant feedback and adaptive joins.

Of course, both of those things were only available if you had columnstore indexes somewhere in near or around the query. So, there were ways to get it legitimately if you had columns run. And then there were some hacky ways to get it regardless.

And those things are being introduced now in SQL Server 2019. And another cool thing that’s coming along in SQL Server 2019 that’s brand new to that product is called Freud. It’s the scalar valued function inlining that our buddy Karthik has been working on brilliantly and doing a brilliant job of.

So, there’s those two things coming. And they are only available if you are in compatibility level 150. Which of course means that if you flip to compatibility level 150, then you are also using that new cardinality estimator.

And that has worked out not so well for a lot of people who I’ve seen start trying to use it. Now, it’s not all the cardinality estimators. Well, they didn’t do testing really.

So, they should have tested before flipping that switch over. But there’s been a lot of consulting calls where someone has said, I’ve upgraded to SQL Server 2014 or 16 or 17. And performance is kind of tanked.

And, you know, when you hop on the phone with them, you start looking at stuff and you say, Oh, well, did you change the compatibility level? And I said, Yeah, of course. We always use the newest one. I said, Well, I have good news for you. With the flip of this one switch, I can restore performance to where it used to be.

Which might not be a good thing or a great thing, but at least you will have the problems that you’re used to on your server. You will not have these brand new problems. So, that’s nice.

But those two features being tied into Compat Level 150 give you a weird choice. It’s the choice between solving potentially a couple big problems with SQL Server. And then also introducing potentially a couple or many more problems by using the new cardinality estimate.

You have to do a lot of very thoughtful, very careful planning to get around that stuff. And what’s crazy is that this is going to be the version, I think, where people have to finally start making those choices. Because not everyone is available to make the kind of changes they need to to get performance right from other perspectives.

So, getting the batch mode stuff and the scalar UDF inlining stuff in is going to be pretty huge. Anyway, that’s it for me. So, Darren says, thanks for the stickers.

Yes, you’re welcome. My pleasure. If you, to anyone who sends me their address, I have Twitter DMs open. If you send me your address, I will send you stickers. I’m happy to do that. Postage costs almost nothing. And I got a very good deal on stickers.

Sticker Mule is a great website for stickers and they often have sales and other things. So, that’s no problem. Darren says, I haven’t looked at CTP3 yet. Did they make any significant changes or have you? I have looked. I did some tweets about new stuff that I found in there.

There was nothing that like blew my mind. There is a weird new use hint. And I’m going to go check on exactly what it’s called so that I don’t get, so I don’t misquote anything here. Excuse me, one of those days I can’t spell select right.

From valid. Is this not valid use hints? And it is called disable loop join avoidance. So, there is a new use like option use hint thing called disable loop join avoidance.

And I’m not exactly sure what it does yet. I have some guesses. I have theorized a couple things. But perhaps it is to make stuff like key lookups more common.

So, there are a lot of queries where you would expect a key lookup to occur and it doesn’t. You get a clustered index scan instead. So, it could be useful for those.

I haven’t put any of these theories to use yet. Mostly because I like having these theories. I dislike having them disproven. Because then I was like, I don’t know what the hell to do. It could also be for columnstore too. Because I know columnstore queries are very apprehensive about doing any kind of key lookups. So, I think it could be for that too.

I don’t know though. I just don’t know yet. I just don’t know yet. We will have to wait and see, won’t we? We will have to eventually wait for Paul White to break out the debugger and figure it out for all of us. Because that is usually what happens, isn’t it?

There is something new or there is something weird going on. And is there anything more comforting or is there anything more reassuring than the words, well, Paul White says. Because you just know that it’s going to be correct. You know that there is a sufficient level of research and care taken in the words that he chooses to share with the rest of us.

That you’re not going to be led astray. So, if there is anything, well, Paul White says. Well, Paul White wrote.

I heard from Paul White. I love it. I love it. It’s a great source. Do we have any other questions? There are eight of you here at least. Are there any other questions?

Nine of you. Wow. Totally watching while at work. Lee, you devil. You dog. Zane says, you may occasionally get your mind broken by Paul White. Yes, that happens quite frequently. It’s a strange thing. What happens when he starts talking.

It’s crazy. It’s like you think that he’s just really good at SQL Server, but he’s good at like many, many things that surround SQL Server as well. And it’s like the stuff that he can also do. So, I’m going to like relate this to like, there are some people who can get good at SQL Server, but it will occupy the entirety of their mind.

Like me, like this is all I can do at once. I can’t have any other hobbies. I can’t have friends.

I can barely walk. And then there are people who like, like SQL Server is like swinging like, like below their weight. It’s like T-ball to them. They can do SQL Server and other stuff. And that’s impressive.

Paul White is one of the people. He can do SQL. He can know a ton about SQL Server and other stuff. It has not pushed all other knowledge out of his brain. And that’s an impressive thing. I have to dedicate everything in this melon. It’s abused melon to SQL Server or else I will not. I will not know any of it.

I will have absolutely nothing. So, it’s always. It’s always fun to watch people who can do this and other things and be good at them. So, yeah, that’s fun stuff, too. All right.

There are a lot of you here. Holy cow. All you playing hooky. No one is at work today. What other questions do we have here? Dan says, are singleton lookups good or bad? I found some indexes that don’t have any seeks or scans in them, but they do have singleton lookups. Singleton lookups just mean that they’re used for key lookups.

They’re not really bad. They’re used for seeks. So, they don’t have any seeks or scans against them, but they do have singleton lookups. All right.

So, yeah. So, usually that just, usually that means that they’re the key, like a primary key or a clustered index or a non-clustered primary key. And they primarily get used for lookup values that are not present in nonclustered indexes. They’re not really a good or bad thing.

I would just peek through the plan cache and I would peek through the plan cache rather. And what I would do is, since you’re using the Blitz procs already, I would say that you should run SP Blitz cache and you should look for a warning called expensive key lookups, which will point to, it will point to if you have an execution plan where a key lookup operation is greater than like 50% of the plan cost, I think. And you could check those out.

So, there are two kinds of key lookups. There are output key lookups where you just go back to the clustered index or the base table if it’s a heap and you just grab columns to show people. So, those are like window dressing.

Those are like columns that you’re just selecting. And then there are predicate key lookups. And predicate key lookups are when you do the same thing except to evaluate a filter. So, if I had, let’s say, an index just on, let’s say I have a clustered index on column one and I have a nonclustered index on column two and my query is like select some column two where column two equals something and column one equals something. I might do a key lookup just to figure out that where clause on column one.

So, for me, fixing the predicate part of key lookups is usually much more important than fixing the output list part. So, unless I’m running into bad parameter sniffing, but working all that stuff out really turns into a very, very long conversation about what exactly is going on with the query. And it’s like, can we rewrite the query?

Do we really need all these columns? Like, is it any frameworks? Is it a sort of a ceiling? Like, there’s a lot of stuff that, like, you need to start working through when you see those things. So, what I would leave at is that there’s nothing inherently wrong with them, but you might want to take a peek through your execution plans and see if you have any expensive key lookups. Again, SV Blitzcache will warn you about that.

And that’s where I would go with it next. Maral says, thoughts on running SSIS and RS services on the same production box as the engine? Yeah, not a fan of that. They take up weird resources and do weird things. I always want to have a different, I always want to have those things on a different server. I’m not crazy about them being on the production box.

Those resources are precious, man. I’ve seen y’all’s production servers. They’re not beefy. They’re not big beefcake servers. And even if they are, you don’t want SSIS and SSRS gobbin’ up the works. It sucks from a licensing perspective, but from a keeping production safe perspective, it is much, much better.

How is my goal of smoking cigarettes in a French graveyard? I am slowly working client by client. I work towards that goal. Client by client.

We’ll see. We’ll see how close we can get. I just need to keep doing it. It would really help if I didn’t live someplace so damn expensive. Oh, man. Yeah.

Yeah. Me and Tara agree on a lot of stuff. That’s why we’re still buddies. At least I hope we are. I don’t know. She might hate me now that I’m consulting competition. She’s working at straight past SQL. And I don’t know. I don’t know.

Maybe we’re at wars. Start licking some shots. Dude, drive by or something. I’m kidding. No, actually, it’s been really nice. Mike Walsh has been great. He’s been offloading. When he has too much work, the poor guy, when he has too much work for his people, he’s been offloading stuff to me.

And so, you know, it’s really nice to have that sort of like, you know, if nothing else is going on, I can, you know, like, hey, Mike, what’d you got? And, you know, have, you know, a few days worth of work to kind of fill up the week, which is always a good thing. So no, no beef there.

No actual beef there. Tara is just very, very busy having to be a remote DBA now. And that’s the lazy consultant. So good for her. That landed in almost the same exact job. That’s a nice, nice touch. Zane posted a link.

Memory management with SSIS and SQL Server can be a large problem. You don’t want to drop pages out of memory because an SSIS package sucks. Yes. And from what I know, most SSIS packages suck. It’s one I’m aware of. Most SSIS packages suck.

Nature of the beast, I suppose. Nature of the beast. Yeah. Not Zane’s own. Zane’s SSIS packages are great. They’re wonderful. Unfortunately, Zane is probably not going to be writing too many SSIS packages anymore.

Mr. Zane Brunette has recently accepted a job as a PFE at Microsoft. He’s going to have the blue badge and the list of trace flags and access to source code and all sorts of other crazy cool things that I’m envious of except for the working at Microsoft part.

I’m very envious of many things that he has going for him except there was a way for me to get those things. And not go to jail, I would do all of those things. And I’m just too happy being by myself.

That’s the problem. I love being alone. It’s wonderful. Love. I love loneliness. Loneliness is great. A lot of people think loneliness is a dirty word. I love it. It’s the best thing in the world. I can do what I want. Wear what I want.

Not wear what I want. Sweatpants at the gym. No two to blame. Yeah. Yeah. Yeah. That’s true. That’s true. I do have to accept the blame for everything. But it’s okay because I’m the only one who notices. I’m very lenient on myself in that regard.

Very lenient. Oh. Yeah, that was fun. So yesterday I was at the gym and I was doing deadlifts because that’s what I usually do. And I was doing some, I was doing like singles, like 475. And I got, well, I was doing doubles at 475 and I got bored.

And I wanted to start doing singles at a higher weight. So I started doing singles at 525 and I videotaped one. And I think it went particularly well. It went like, it was just like, it went up like crazy. You can barely see what’s going on there.

But man, I was very happy with how quick it went up. It’s going, it’s going, it’s going. Ah, come on. Lift it dummy. Why are you so slow? Big fellow walking behind me.

Yeah, that went all up. That went all the way up. I was very happy with that lift. And then I did, I did a few more of those and I was dead. I still think I’m dead. That’s why I’m so hot and hangry today is because it’s tough to recover from doing that. It’s hard.

It’s hard. Your central nervous system is just like yelling at you. It’s like you are the worst. Why do you do that? I don’t know. I don’t know. Mostly just to impress myself. No one else is impressed.

Ah, man. All right. Come on. Give me a, give me a question here. People can’t not have any questions. Why else would you show up to Q and A? You didn’t have questions. You can’t just, you can’t just be here to watch me. I’m not that, I’m not that excited. I’m not that interested.

I’m not that interested. Come on. At least your forehead doesn’t explode like half the horn. Yeah, that’s true. It comes close though. If I, if I strain hard enough, I can get, so I can get, I get this like split vein thing right here.

It looks like, like the flux capacitor. It’s crazy. Uh, yeah. Rogue plates. Can’t do it. If I don’t have, I mean, what other plates do you buy? The only thing that sucks about the gym that I go to is they don’t have like the, like the, like real Ollie, like thin plates.

It’s all like the thick, the thick rubbery ones, not like the thin and ones. So like, if you have like six blue plates on the bar, that’s all you can do. Like, and then if you want to do like more, you have to like take off some of like the good rogue plates and put on like some of the, the metal octagon plates. And that’s, I mean, it’s, it’s not that they’re bad.

It’s just, you know, it breaks up. It breaks up how cool it looks. And when you’re OCD like me, and you just want to have a bar, same, like, damn it. Give me, do this. Let’s see the thoughts on diagnosing network connectivity issues other than trying to tell net via command prompt or using the deck.

So is it like you set up a SQL Server and you just can’t connect to it off the bat or like queries are timing out getting weird? Cause it’s like, you just set up a SQL Server. The first things I always do is check firewall rules.

If it’s a named instance, I make sure that the browser service is running. I’ve also learned to very, very, very carefully check the instance names. Because there, there was a, there was an, there was an instance around the time. Oh, I don’t know when CTP three came out when I spent 20 minutes trying to connect to the wrong server name.

So that was, that was great. Random users and apps can’t connect. The most common thing, most common reason that I’ve seen for random users and apps not being able to connect is thread pool.

Thread pool is when you run out of worker threads and you can no longer assign even like a session ID to a query. And that’s a pretty bad thing. So what I would do is if you wanted to run like SP blitz on your server, you might get a warning about thread pool weights.

If you don’t have that or can’t run that, then you can, you can run pretty much just any weight stat aggregating script and see if you have thread pool weights on there. Like lining up because that’s usually a sign that bad things are happening. Thread pool generally happens if like you have a bunch of parallel queries coming in and trying to run stuff at once.

Or if you have like a big blocking chain or something and you know, stuff like if you have a monitoring tool, you might see big blank spots in the monitoring tool too, because that thing can’t, that thing needs threads too. And it can’t connect. So there’s a lot of stuff, a lot of reasons why that might happen, but thread pool is among chief among them.

Stop naming your instances. Oh, yeah. Woo woo. Stop naming your instances. Weird things. I named my instance CTP and I thought that I had named it CTP two in the past.

And so I was trying to connect to CTP two and I couldn’t. And I guess kept trying to like, damn it, what’s going on? And I was like, oh yeah, it’s just CTP. No two. And I did that.

Everything worked and it was magical. I felt it was a big, big, big win for me there. It’s like amazing. Let’s, let’s sanity check. I never sanity check first. It’s always like the fourth or fifth thing I do. And then I feel dumb because I’m like, through all these years, why isn’t that the first thing that you do? Like, I’m not crazy.

I can’t be crazy. I know I did this right. Oh, yeah. Dan has a good point. I do have a script. Now you’re going to make me go find that. Let’s see if I can remember how to type.

There we go. I can even get to, I can even get to my own website. Paul Randall even said, this is a nice script.

If you check the comments, you check the comments. There’s a thing that says from Paul, comment from Paul Randall that says, nice script. And then I felt bad because like a week later, someone posted a question about wait stats on the DBA stack exchange site. And they were like, I ran this script and they paid they pasted the script in but it was like just the script was like just the queries from the script and they were like, I ran this script.

And it’s telling me that I have like 86% CX packet or something. And I was like 86% of what in my head. I’m like, damn it.

Why do you do? And so like I write this answer. I’m like, the reason why that script stinks is because it doesn’t tell you like what the uptime of the server is. You don’t know what 86% of, you know, like, like if you have like, you know, like three days of CX packet weights, but your server’s only been up for a half hour. That’s way different than if like you have three days of CX pack away.

So it serves enough for a month. And I didn’t like, cause you know, working for Brent for the last four years, I didn’t, I haven’t spent a lot of time looking at other people’s scripts. It was always very much like, you know, write scripts from scratch, like do stuff on your own. Don’t look at other people’s stuff.

You don’t want to like get, get caught like influencing or have anyone say you stole that script I posted on the blog. You suck. And so I was like, okay, no problem. I’ll write everything. So I don’t remember what, what Paula scripts looked like. And so like, like a few hours later on my answer, there was, there was like, there was a comment from Paul.

And that’s when, that’s when I started thinking, I was like, why would Paul randomly comment on this, this answer? And then I went and I looked and I was like, oh shit, that’s a script. It’s like, eh.

So I know he’s not watching, but sorry about that, Paul. Honest mistake. It wasn’t, it wasn’t really, I wasn’t trying to make fun of you. I was just trying to help someone. Anyway, I can’t remember if I’ve told that story on here before.

I remember telling someone that story before. I forget, I forget when it was. Anyway, could have been here. It could have not been. Don’t tell me if it was, cause I don’t want to get embarrassed. Don’t tell me. Don’t tell me anything. Don’t tell me when I forget things. There’s nothing more embarrassing than forgetting things.

Let’s see. Anything on Twitter. No. So I, on Twitter, I made a prediction that. If SQL Server 2019 does not increase the RAM cap. It really should increase the RAM cap to 256 gigs of RAM.

128 is nothing these days. Nothing. My desktop, this thing that I can kick. Willy nilly has 128 gigs of RAM. There’s no reason to expect. A.

Production SQL Server instance, even standard edition to be capped to 128 gigs of RAM. It also helps because there are very strange instance sizes up in the cloud. They’re very oddly sized with the CPU to RAM ratio. And I think.

There are, there are many situations. There are many servers that’s see that standard edition would make sense on if it could have 256. Gigs of memory, but don’t make sense. Um, because it only accepts 128. Like there are some 16 core instances up there. When, or like 16 to like 24 core instances up there that have.

Like 256. Or like 200 and some 66. Some like weird number gigs of memory, which is, which sounds great. And you’re like, man, standard edition would be cool to put on there, but. I would have.

A hundred or so gigs of RAM. Sitting around. No, I, I, I get that. The differentiation, but they’re like that number is the cap on the buffer pool and you can use stuff above the buffer pool. I just don’t see a lot of people who are like, I only need to cash. 228 gigs of memory of stuff in the buffer pool, but boy, howdy.

I sure wish I had a hundred gigs for memory grants and the plan cash and other stuff. I just, I don’t, I don’t meet those people terribly often. So I think that having standard edition go to 256 gigs of memory would be an excellent thing. But anyway, I was saying that my prediction was if it doesn’t have, if they don’t increase the RAM cap or if they don’t make the scale or UDF inlining or what do you call it?

Batch mode for rowstore available on standard edition, then I think adoption is going to suck. Going to continue to stifle adoption because the salespeople are jerks.

They really got to cut that crap out. Not helping anyone. Not doing yourself any favors. Not, you know, you’re not selling more enterprise licenses by doing that. You’re keeping more people on the old licenses as they already have.

People just refuse to upgrade. I know I knew people on SQL Server 2008 standard edition who would not upgrade to 2008 R2 because in 2008 there was no cap on memory. 2008 R2 is when standard edition got the 64 gig cap. So there are people who, clients who I knew stayed on 2008.

Probably still on 2008 because there was no cap on memory. They could put in as much as they wanted. No. I don’t know. That’s what I do.

But, you know, I’m free spirited. Might not fit in well with that corporate culture. Anyway, we’re about at the half hour mark. It’s very hot. It’s very hot.

Very hot in here. And I want to stop being hot. So I’m going to get these headphones off and step away and sit down and drink some water. Anyway, thanks for joining. I hope you at least had a good time. Anyway, I’ll catch you next week because as far as I know I’ll still be here.

It’s amazing. Friday after Friday. It’s nuts. Goodbye.

Video Summary

In this video, I dive into some of the exciting new features coming in SQL Server 2019, particularly focusing on batch mode memory grant feedback and adaptive joins. These features are only available if you’re using compatibility level 150, which presents a unique set of considerations for database administrators. I also delve into key lookups, explaining their role in execution plans and how to identify potentially expensive ones using the SQL Server Blitz cache procedures. The discussion then shifts to practical advice on running SSIS and Reporting Services (SSRS) services alongside the SQL Server engine, emphasizing the importance of keeping production environments as clean and efficient as possible. Throughout the video, I share some personal insights, including my ongoing journey towards achieving a long-held goal and reflecting on recent changes in my professional landscape, such as the exciting news about Zane Brunette joining Microsoft. The conversation also touches on the nature of working alone versus being part of a team, with a humorous nod to my gym routine and its impact on my current state of mind—hot, hangry, and ready for some serious database tuning challenges!

Full Transcript

It is so hot. It’s so hot and I’m so annoyed with all the heat. I finally got to eat lunch. Prior to that, I was both hot and hangry at the same time. My hair is sweaty, I’m sorry. I look like garbage. I look like garbage. It’s not fun. It’s not fun. I should probably mention that I’m doing this somewhere. I don’t think I did that. The one person who is here. Thank you for showing up. Thank you for having faith in me. One person who is here. Hopefully you’re not just some Russian bot.

Here to collude. Here to collude with me about things. Oh, and you’re gone. Goodbye, Russian bot. Goodbye forever. Do-do-do-do. There, I’ve tweeted about it now, so no one has an excuse for not showing up.

So, if no one shows up on this beautiful Friday, it’s a hot Friday, then I’m going to talk about something that’s been on my mind lately. And that something is the coming… Off to a good start. I’m not going to say the word coming correctly. The coming reckoning. That’s where a coming came from, was coming reckoning.

The coming reckoning that is due with SQL Server 2019. So, there were things introduced in SQL Server 2017 that I thought were really cool. And that was batch mode memory grant feedback and adaptive joins.

Of course, both of those things were only available if you had columnstore indexes somewhere in near or around the query. So, there were ways to get it legitimately if you had columns run. And then there were some hacky ways to get it regardless.

And those things are being introduced now in SQL Server 2019. And another cool thing that’s coming along in SQL Server 2019 that’s brand new to that product is called Freud. It’s the scalar valued function inlining that our buddy Karthik has been working on brilliantly and doing a brilliant job of.

So, there’s those two things coming. And they are only available if you are in compatibility level 150. Which of course means that if you flip to compatibility level 150, then you are also using that new cardinality estimator.

And that has worked out not so well for a lot of people who I’ve seen start trying to use it. Now, it’s not all the cardinality estimators. Well, they didn’t do testing really.

So, they should have tested before flipping that switch over. But there’s been a lot of consulting calls where someone has said, I’ve upgraded to SQL Server 2014 or 16 or 17.

And performance is kind of tanked. And, you know, when you hop on the phone with them, you start looking at stuff and you say, Oh, well, did you change the compatibility level?

And I said, Yeah, of course. We always use the newest one. I said, Well, I have good news for you. With the flip of this one switch, I can restore performance to where it used to be. Which might not be a good thing or a great thing, but at least you will have the problems that you’re used to on your server.

You will not have these brand new problems. So, that’s nice. But those two features being tied into Compat Level 150 give you a weird choice.

It’s the choice between solving potentially a couple big problems with SQL Server. And then also introducing potentially a couple or many more problems by using the new cardinality estimate. You have to do a lot of very thoughtful, very careful planning to get around that stuff.

And what’s crazy is that this is going to be the version, I think, where people have to finally start making those choices. Because not everyone is available to make the kind of changes they need to to get performance right from other perspectives. So, getting the batch mode stuff and the scalar UDF inlining stuff in is going to be pretty huge.

Anyway, that’s it for me. So, Darren says, thanks for the stickers. Yes, you’re welcome.

My pleasure. If you, to anyone who sends me their address, I have Twitter DMs open. If you send me your address, I will send you stickers. I’m happy to do that. Postage costs almost nothing.

And I got a very good deal on stickers. Sticker Mule is a great website for stickers and they often have sales and other things. So, that’s no problem.

Darren says, I haven’t looked at CTP3 yet. Did they make any significant changes or have you? I have looked. I did some tweets about new stuff that I found in there. There was nothing that like blew my mind.

There is a weird new use hint. And I’m going to go check on exactly what it’s called so that I don’t get, so I don’t misquote anything here. Excuse me, one of those days I can’t spell select right.

From valid. Is this not valid use hints? And it is called disable loop join avoidance.

So, there is a new use like option use hint thing called disable loop join avoidance. And I’m not exactly sure what it does yet. I have some guesses.

I have theorized a couple things. But perhaps it is to make stuff like key lookups more common. So, there are a lot of queries where you would expect a key lookup to occur and it doesn’t.

You get a clustered index scan instead. So, it could be useful for those. I haven’t put any of these theories to use yet.

Mostly because I like having these theories. I dislike having them disproven. Because then I was like, I don’t know what the hell to do. It could also be for columnstore too. Because I know columnstore queries are very apprehensive about doing any kind of key lookups.

So, I think it could be for that too. I don’t know though. I just don’t know yet. I just don’t know yet.

We will have to wait and see, won’t we? We will have to eventually wait for Paul White to break out the debugger and figure it out for all of us. Because that is usually what happens, isn’t it?

There is something new or there is something weird going on. And is there anything more comforting or is there anything more reassuring than the words, well, Paul White says. Because you just know that it’s going to be correct.

You know that there is a sufficient level of research and care taken in the words that he chooses to share with the rest of us. That you’re not going to be led astray. So, if there is anything, well, Paul White says.

Well, Paul White wrote. I heard from Paul White. I love it.

I love it. It’s a great source. Do we have any other questions? There are eight of you here at least. Are there any other questions? Nine of you. Wow.

Totally watching while at work. Lee, you devil. You dog. Zane says, you may occasionally get your mind broken by Paul White. Yes, that happens quite frequently. It’s a strange thing.

What happens when he starts talking. It’s crazy. It’s like you think that he’s just really good at SQL Server, but he’s good at like many, many things that surround SQL Server as well. And it’s like the stuff that he can also do.

So, I’m going to like relate this to like, there are some people who can get good at SQL Server, but it will occupy the entirety of their mind. Like me, like this is all I can do at once. I can’t have any other hobbies.

I can’t have friends. I can barely walk. And then there are people who like, like SQL Server is like swinging like, like below their weight. It’s like T-ball to them.

They can do SQL Server and other stuff. And that’s impressive. Paul White is one of the people. He can do SQL. He can know a ton about SQL Server and other stuff. It has not pushed all other knowledge out of his brain.

And that’s an impressive thing. I have to dedicate everything in this melon. It’s abused melon to SQL Server or else I will not. I will not know any of it.

I will have absolutely nothing. So, it’s always. It’s always fun to watch people who can do this and other things and be good at them. So, yeah, that’s fun stuff, too.

All right. There are a lot of you here. Holy cow. All you playing hooky. No one is at work today. What other questions do we have here?

Dan says, are singleton lookups good or bad? I found some indexes that don’t have any seeks or scans in them, but they do have singleton lookups. Singleton lookups just mean that they’re used for key lookups.

They’re not really bad. They’re used for seeks. So, they don’t have any seeks or scans against them, but they do have singleton lookups. All right.

So, yeah. So, usually that just, usually that means that they’re the key, like a primary key or a clustered index or a non-clustered primary key. And they primarily get used for lookup values that are not present in nonclustered indexes.

They’re not really a good or bad thing. I would just peek through the plan cache and I would peek through the plan cache rather. And what I would do is, since you’re using the Blitz procs already, I would say that you should run SP Blitz cache and you should look for a warning called expensive key lookups, which will point to, it will point to if you have an execution plan where a key lookup operation is greater than like 50% of the plan cost, I think.

And you could check those out. So, there are two kinds of key lookups. There are output key lookups where you just go back to the clustered index or the base table if it’s a heap and you just grab columns to show people.

So, those are like window dressing. Those are like columns that you’re just selecting. And then there are predicate key lookups.

And predicate key lookups are when you do the same thing except to evaluate a filter. So, if I had, let’s say, an index just on, let’s say I have a clustered index on column one and I have a nonclustered index on column two and my query is like select some column two where column two equals something and column one equals something. I might do a key lookup just to figure out that where clause on column one.

So, for me, fixing the predicate part of key lookups is usually much more important than fixing the output list part. So, unless I’m running into bad parameter sniffing, but working all that stuff out really turns into a very, very long conversation about what exactly is going on with the query. And it’s like, can we rewrite the query?

Do we really need all these columns? Like, is it any frameworks? Is it a sort of a ceiling? Like, there’s a lot of stuff that, like, you need to start working through when you see those things. So, what I would leave at is that there’s nothing inherently wrong with them, but you might want to take a peek through your execution plans and see if you have any expensive key lookups.

Again, SV Blitzcache will warn you about that. And that’s where I would go with it next. Maral says, thoughts on running SSIS and RS services on the same production box as the engine?

Yeah, not a fan of that. They take up weird resources and do weird things. I always want to have a different, I always want to have those things on a different server. I’m not crazy about them being on the production box.

Those resources are precious, man. I’ve seen y’all’s production servers. They’re not beefy. They’re not big beefcake servers. And even if they are, you don’t want SSIS and SSRS gobbin’ up the works.

It sucks from a licensing perspective, but from a keeping production safe perspective, it is much, much better. How is my goal of smoking cigarettes in a French graveyard? I am slowly working client by client.

I work towards that goal. Client by client. We’ll see. We’ll see how close we can get. I just need to keep doing it. It would really help if I didn’t live someplace so damn expensive.

Oh, man. Yeah. Yeah.

Me and Tara agree on a lot of stuff. That’s why we’re still buddies. At least I hope we are. I don’t know. She might hate me now that I’m consulting competition. She’s working at straight past SQL. And I don’t know.

I don’t know. Maybe we’re at wars. Start licking some shots. Dude, drive by or something. I’m kidding.

No, actually, it’s been really nice. Mike Walsh has been great. He’s been offloading. When he has too much work, the poor guy, when he has too much work for his people, he’s been offloading stuff to me. And so, you know, it’s really nice to have that sort of like, you know, if nothing else is going on, I can, you know, like, hey, Mike, what’d you got?

And, you know, have, you know, a few days worth of work to kind of fill up the week, which is always a good thing. So no, no beef there. No actual beef there.

Tara is just very, very busy having to be a remote DBA now. And that’s the lazy consultant. So good for her. That landed in almost the same exact job. That’s a nice, nice touch.

Zane posted a link. Memory management with SSIS and SQL Server can be a large problem. You don’t want to drop pages out of memory because an SSIS package sucks. Yes.

And from what I know, most SSIS packages suck. It’s one I’m aware of. Most SSIS packages suck. Nature of the beast, I suppose.

Nature of the beast. Yeah. Not Zane’s own. Zane’s SSIS packages are great. They’re wonderful. Unfortunately, Zane is probably not going to be writing too many SSIS packages anymore.

Mr. Zane Brunette has recently accepted a job as a PFE at Microsoft. He’s going to have the blue badge and the list of trace flags and access to source code and all sorts of other crazy cool things that I’m envious of except for the working at Microsoft part.

I’m very envious of many things that he has going for him except there was a way for me to get those things. And not go to jail, I would do all of those things. And I’m just too happy being by myself.

That’s the problem. I love being alone. It’s wonderful. Love. I love loneliness. Loneliness is great. A lot of people think loneliness is a dirty word. I love it.

It’s the best thing in the world. I can do what I want. Wear what I want. Not wear what I want. Sweatpants at the gym. No two to blame.

Yeah. Yeah. Yeah. That’s true. That’s true. I do have to accept the blame for everything. But it’s okay because I’m the only one who notices. I’m very lenient on myself in that regard.

Very lenient. Oh. Yeah, that was fun. So yesterday I was at the gym and I was doing deadlifts because that’s what I usually do. And I was doing some, I was doing like singles, like 475.

And I got, well, I was doing doubles at 475 and I got bored. And I wanted to start doing singles at a higher weight. So I started doing singles at 525 and I videotaped one.

And I think it went particularly well. It went like, it was just like, it went up like crazy. You can barely see what’s going on there. But man, I was very happy with how quick it went up.

It’s going, it’s going, it’s going. Ah, come on. Lift it dummy. Why are you so slow? Big fellow walking behind me.

Yeah, that went all up. That went all the way up. I was very happy with that lift. And then I did, I did a few more of those and I was dead. I still think I’m dead.

That’s why I’m so hot and hangry today is because it’s tough to recover from doing that. It’s hard. It’s hard.

Your central nervous system is just like yelling at you. It’s like you are the worst. Why do you do that? I don’t know. I don’t know. Mostly just to impress myself.

No one else is impressed. Ah, man. All right. Come on. Give me a, give me a question here. People can’t not have any questions. Why else would you show up to Q and A?

You didn’t have questions. You can’t just, you can’t just be here to watch me. I’m not that, I’m not that excited. I’m not that interested. I’m not that interested. Come on.

At least your forehead doesn’t explode like half the horn. Yeah, that’s true. It comes close though. If I, if I strain hard enough, I can get, so I can get, I get this like split vein thing right here.

It looks like, like the flux capacitor. It’s crazy. Uh, yeah. Rogue plates. Can’t do it. If I don’t have, I mean, what other plates do you buy? The only thing that sucks about the gym that I go to is they don’t have like the, like the, like real Ollie, like thin plates.

It’s all like the thick, the thick rubbery ones, not like the thin and ones. So like, if you have like six blue plates on the bar, that’s all you can do. Like, and then if you want to do like more, you have to like take off some of like the good rogue plates and put on like some of the, the metal octagon plates.

And that’s, I mean, it’s, it’s not that they’re bad. It’s just, you know, it breaks up. It breaks up how cool it looks.

And when you’re OCD like me, and you just want to have a bar, same, like, damn it. Give me, do this. Let’s see the thoughts on diagnosing network connectivity issues other than trying to tell net via command prompt or using the deck.

So is it like you set up a SQL Server and you just can’t connect to it off the bat or like queries are timing out getting weird? Cause it’s like, you just set up a SQL Server. The first things I always do is check firewall rules.

If it’s a named instance, I make sure that the browser service is running. I’ve also learned to very, very, very carefully check the instance names. Because there, there was a, there was an, there was an instance around the time.

Oh, I don’t know when CTP three came out when I spent 20 minutes trying to connect to the wrong server name. So that was, that was great. Random users and apps can’t connect.

The most common thing, most common reason that I’ve seen for random users and apps not being able to connect is thread pool. Thread pool is when you run out of worker threads and you can no longer assign even like a session ID to a query. And that’s a pretty bad thing.

So what I would do is if you wanted to run like SP blitz on your server, you might get a warning about thread pool weights. If you don’t have that or can’t run that, then you can, you can run pretty much just any weight stat aggregating script and see if you have thread pool weights on there. Like lining up because that’s usually a sign that bad things are happening.

Thread pool generally happens if like you have a bunch of parallel queries coming in and trying to run stuff at once. Or if you have like a big blocking chain or something and you know, stuff like if you have a monitoring tool, you might see big blank spots in the monitoring tool too, because that thing can’t, that thing needs threads too. And it can’t connect.

So there’s a lot of stuff, a lot of reasons why that might happen, but thread pool is among chief among them. Stop naming your instances. Oh, yeah. Woo woo.

Stop naming your instances. Weird things. I named my instance CTP and I thought that I had named it CTP two in the past. And so I was trying to connect to CTP two and I couldn’t.

And I guess kept trying to like, damn it, what’s going on? And I was like, oh yeah, it’s just CTP. No two. And I did that. Everything worked and it was magical.

I felt it was a big, big, big win for me there. It’s like amazing. Let’s, let’s sanity check. I never sanity check first. It’s always like the fourth or fifth thing I do. And then I feel dumb because I’m like, through all these years, why isn’t that the first thing that you do?

Like, I’m not crazy. I can’t be crazy. I know I did this right. Oh, yeah.

Dan has a good point. I do have a script. Now you’re going to make me go find that. Let’s see if I can remember how to type. There we go.

I can even get to, I can even get to my own website. Paul Randall even said, this is a nice script. If you check the comments, you check the comments.

There’s a thing that says from Paul, comment from Paul Randall that says, nice script. And then I felt bad because like a week later, someone posted a question about wait stats on the DBA stack exchange site. And they were like, I ran this script and they paid they pasted the script in but it was like just the script was like just the queries from the script and they were like, I ran this script.

And it’s telling me that I have like 86% CX packet or something. And I was like 86% of what in my head. I’m like, damn it.

Why do you do? And so like I write this answer. I’m like, the reason why that script stinks is because it doesn’t tell you like what the uptime of the server is. You don’t know what 86% of, you know, like, like if you have like, you know, like three days of CX packet weights, but your server’s only been up for a half hour. That’s way different than if like you have three days of CX pack away.

So it serves enough for a month. And I didn’t like, cause you know, working for Brent for the last four years, I didn’t, I haven’t spent a lot of time looking at other people’s scripts. It was always very much like, you know, write scripts from scratch, like do stuff on your own.

Don’t look at other people’s stuff. You don’t want to like get, get caught like influencing or have anyone say you stole that script I posted on the blog. You suck.

And so I was like, okay, no problem. I’ll write everything. So I don’t remember what, what Paula scripts looked like. And so like, like a few hours later on my answer, there was, there was like, there was a comment from Paul.

And that’s when, that’s when I started thinking, I was like, why would Paul randomly comment on this, this answer? And then I went and I looked and I was like, oh shit, that’s a script. It’s like, eh.

So I know he’s not watching, but sorry about that, Paul. Honest mistake. It wasn’t, it wasn’t really, I wasn’t trying to make fun of you.

I was just trying to help someone. Anyway, I can’t remember if I’ve told that story on here before. I remember telling someone that story before.

I forget, I forget when it was. Anyway, could have been here. It could have not been. Don’t tell me if it was, cause I don’t want to get embarrassed. Don’t tell me. Don’t tell me anything.

Don’t tell me when I forget things. There’s nothing more embarrassing than forgetting things. Let’s see. Anything on Twitter. No. So I, on Twitter, I made a prediction that. If SQL Server 2019 does not increase the RAM cap.

It really should increase the RAM cap to 256 gigs of RAM. 128 is nothing these days. Nothing.

My desktop, this thing that I can kick. Willy nilly has 128 gigs of RAM. There’s no reason to expect. A.

Production SQL Server instance, even standard edition to be capped to 128 gigs of RAM. It also helps because there are very strange instance sizes up in the cloud. They’re very oddly sized with the CPU to RAM ratio.

And I think. There are, there are many situations. There are many servers that’s see that standard edition would make sense on if it could have 256. Gigs of memory, but don’t make sense.

Um, because it only accepts 128. Like there are some 16 core instances up there. When, or like 16 to like 24 core instances up there that have.

Like 256. Or like 200 and some 66. Some like weird number gigs of memory, which is, which sounds great. And you’re like, man, standard edition would be cool to put on there, but.

I would have. A hundred or so gigs of RAM. Sitting around. No, I, I, I get that.

The differentiation, but they’re like that number is the cap on the buffer pool and you can use stuff above the buffer pool. I just don’t see a lot of people who are like, I only need to cash. 228 gigs of memory of stuff in the buffer pool, but boy, howdy.

I sure wish I had a hundred gigs for memory grants and the plan cash and other stuff. I just, I don’t, I don’t meet those people terribly often. So I think that having standard edition go to 256 gigs of memory would be an excellent thing.

But anyway, I was saying that my prediction was if it doesn’t have, if they don’t increase the RAM cap or if they don’t make the scale or UDF inlining or what do you call it? Batch mode for rowstore available on standard edition, then I think adoption is going to suck.

Going to continue to stifle adoption because the salespeople are jerks. They really got to cut that crap out. Not helping anyone. Not doing yourself any favors.

Not, you know, you’re not selling more enterprise licenses by doing that. You’re keeping more people on the old licenses as they already have. People just refuse to upgrade.

I know I knew people on SQL Server 2008 standard edition who would not upgrade to 2008 R2 because in 2008 there was no cap on memory. 2008 R2 is when standard edition got the 64 gig cap.

So there are people who, clients who I knew stayed on 2008. Probably still on 2008 because there was no cap on memory. They could put in as much as they wanted.

No. I don’t know. That’s what I do. But, you know, I’m free spirited.

Might not fit in well with that corporate culture. Anyway, we’re about at the half hour mark. It’s very hot.

It’s very hot. Very hot in here. And I want to stop being hot. So I’m going to get these headphones off and step away and sit down and drink some water. Anyway, thanks for joining.

I hope you at least had a good time. Anyway, I’ll catch you next week because as far as I know I’ll still be here. It’s amazing.

Friday after Friday. It’s nuts. Goodbye.

Going Further


If this is the kind of SQL Server stuff you love learning about, you’ll love my training. Blog readers get 25% off the Everything Bundle — over 100 hours of performance tuning content. Need hands-on help? I offer consulting engagements from targeted investigations to ongoing retainers. Want a quick sanity check before committing to a full engagement? Schedule a call — no commitment required.

Live SQL Server Q&A!

ICYMI


Last week’s thrilling, stunning, flawless episode of whatever-you-wanna-call-it.

Video Summary

In this video, I delve into some fundamental SQL Server and database design principles. Starting off, I discuss the importance of avoiding wide tables—tables with more than 100 columns—and how they can lead to indexing challenges and maintenance headaches. I also share a few patterns to look out for in your schema that might indicate poorly designed tables, such as prefixing column names or using numeric suffixes for similar data. Additionally, I explore the concept of Michael J. Swartz’s 10% rule, emphasizing that while SQL Server supports up to 1,024 columns per table, it’s wise to keep your tables narrow and well-structured.

Moving on, I cover memory grants in queries, explaining how they are necessary for operations like sorting and hashing but not always required. I provide practical examples of scenarios where a query can process terabytes of data without needing any memory grant, highlighting the importance of understanding these nuances for optimizing performance. Throughout the video, I offer actionable advice on improving your database design, referencing Lewis Davidson’s book “Relational Database Design for SQL Server” as an excellent resource to guide developers in creating more efficient and maintainable databases.

Full Transcript

So, let’s see. Science and technology is the, I don’t know, channel, stream that this thing takes place in. I don’t know if that’s really accurate. I don’t know if I’d call it science or technology. Maybe, maybe neither. It’s not terribly scientific. And aside from the fact that I use a computer to broadcast, I have absolutely no technology in here. I did get a new phone. Got a Pixel 3 XL. Pretty psyched about that. It’s a picture of my kid picking her nose on there. She’s a good time. So, just to kill a little time until questions come in, I see people in here, which is very exciting. Thank you for joining me. I’ve been writing some blog posts about my first six months of consulting independently. And, I don’t know, it was a real tear-jerker. There were some emotional moments in there. I’m kidding.

I replaced feelings with whiskey many decades ago. But, anyway, I’ve been, shut up, motorcycle. Back to your biker gang. I’ve been getting allergy shots for, I want to say, like, three years. At least three years. I’m on, like, the monthly maintenance shots now where, like, I just, like, they don’t even make me safe for a half hour anymore. I get my shot and I hang out for, like, 10, 15 and just say goodbye. Whiskey. Whiskey. Tisigr asks, whiskey or bourbon? I would never drink bourbon. Bourbon is just hyped up maple syrup. I would never drink bourbon. I’m a scotch guy.

And, like, very specifically, I like Iowa scotches. I like the stuff that tastes like burning band-aids. That’s my jam. But, yeah, we’ve been getting allergy shots for three years now, monthly maintenance. And the last, I don’t know what changed in the, I don’t know what changed in the world. I don’t know what new life form showed up on planet Earth. But it’s, like, I’ve never had a shot in my life. For, like, a long time, they were great.

I, like, I went from having, like, severe, like, eyes running, nose constantly bubbling, gross stuff, allergies, like, unable to function allergies, to, like, I would take maybe, like, antihistamine, like, once a week or so or, like, once every couple weeks. It was great. But, man, the last week has been absolutely positively brutal. Like, I’ve been taking stuff every day. Like, every morning, I wake up at, like, 3, 4 in the morning with, like, my face in awful condition.

I don’t know what the hell is different this year. But, man, it is bad. Bad. Ugh. It’s terrible.

Anyway, I don’t know. I don’t know. Someone ask a question. There are at least six of you here.

I don’t know how many more are going to come in. There are some of you. Someone has to have a SQL Server question. You can’t just come here to hear me complain about things or blab on and on about blog posts and whatnot.

Someone has to have a question about SQL Server. Please, God, someone have a question. Come on, there’s, like, 10 of you. Laura says, do you have any thoughts on replacing temp tables with in-memory OLTP?

Yeah, don’t. Hecaton is like herpes. It doesn’t have the nerve to kill you.

It just hangs around being awful for the rest of your life and flaring up at inappropriate times. So, I just have not found anything compelling about Hecaton. It seems like every time…

And this is not just me. This is some very smart people that I’m friends with. Every time they think that they have a specific problem with latches that Hecaton might solve, it is an absolute dead end.

Absolute dead end. In SQL Server 2019, TempDB is going to use… Well, you can use TempDB as, like, an Hecaton-y thing anyway.

I would pry this whole lot to see if that happens. But, whereas, I guess the question is, why? Why?

What are you trying to fix? What is a problem that we’re trying to solve here? What do you think Hecaton is going to make better in your stored procedure that you want to use in memory for?

That’s the big question. What’s going to get better about it? What’s going to get better? I don’t know.

Do, do. Do, do. Do, do. Try to have better indexing of… What on earth? You can index TempTables now.

What’s missing from your current TempTable indexing that Hecaton is going to provide a safe and secure solution for? That’s what I’m curious about.

Too slow. Well, so here’s the thing. What’s too slow? Creating the index or populating the Temp…

or populating the table? Josh says Hecaton indexes are extra confusing. Yeah, they are. So who knows how many hash buckets you should set up? Creating the index.

So… Creating the index is too slow. Okay. Fair enough. See, this is one of those things where it’s like, we’re going to end up going down a real rabbit hole.

Because I’m going to ask you what kind of index. And what kind of data types you’re indexing. And how many rows it is. And a lot of other stuff.

And this is… I hope you’re prepared. Because this can go on for a long, long time. I know.

Then you’d have to actually look at code. Bad news is you’d actually have to look at code to implement Hecaton. and get it set up and running there. If you don’t…

If creating an index is slow now on a Temp table, I don’t think it’s going to be any faster in Hecaton. It’s not free.

Nothing is free. Nothing is free. Yeah, no. Well, you know, my…

So Hecaton is very specifically designed to deal with locking and lashing issues. The in-memory portion, I think, is…

The misgiving around it is that it’ll make any workload faster. And that’s not really true. And the use cases where I’ve seen Hecaton be successful is with really large-scale, fast data ingestion into tables where data is not going to live for very long.

So the example that I always give, because it’s the example that I’ve seen work best, was with online gambling, where data had to come in very fast. And we cared about that data for a very short amount of time, where we wanted very fast updates and being able to get to that data to be snappy and not get blocked up and locked.

And then after like an hour or so, or whenever the betting thing is over, they get pushed out to regular on this table. So that’s the only time.

It’s the only time I’ve ever seen Hecaton be successful. For every other weird niche thing that someone’s like, oh, I bet if we just did this in memory, it’d be faster.

It has never worked out. Never worked out. I don’t know. It’s like, people hear about these features and they, I don’t know, the pamphlets that Microsoft comes up with for these things are amazing because they make it seem like they’re going to fix every single problem that you’re having.

They just sound like this, like golden acres retirement home for your data. And geez, never, never seems to go.

Yeah. So like with the Lars, I guess to put it, put it as succinctly as possible. you can try it, but be very careful with the database that you try it in because once you have created an in-memory file group in a database, one cannot drop an in-memory file group.

One must drop and recreate the database. So if you’re going to try it somewhere, create a new database and enable things there. Don’t do it with a database you care about.

So that’s about all I got. Let’s see. TZH X says, do you have a rule of thumb for how wide a table is too wide? Currently have a bunch of core tables in the system with a hundred plus columns each.

Yeah. You’re about there. So I think one way to, one way to phrase this really well is the maximum number of columns you can have in a table is a thousand and 24.

And a really smart friend of mine, Canadian fella, says it has, has sort of a, what do you call is it? Michael’s Michael J. Swartz 10% rule with, if you’re using 10% of a maximum of a limit, right?

Cause these are limits. They are not goals. They’re not things that you are trying to attain. They are things that they are, they are like, like the, the capacity at which SQL Server will stop functioning. So you’re, you’re right about there.

And the thing that really sucks about a hundred column tables is they’re really, really difficult to index. Well, unless you don’t care about read speeds from them, unless you’re just dumping data in there and not really doing anything with it, then it becomes really hard to index that.

Cause people are, people are going to you know, they’re going to want to search on, on weird sets of columns. And they’re going to want to select weird sets of columns and potentially order by weird sets of columns.

And all that adds up to is a lot of a heartache and pain and trying to index those things. So when I start seeing tables like that, that there are three patterns that I look out for very specifically.

one is column names that have prefixes. So like customer name, customer number, customer address, customer, things like that. Cause those should all be in a table called customer. The other thing that I look out for is, uh, columns that end in numbers.

So like phone one, phone two, phone three, email one, email two, email three. I’m also, I also pay attention to those. Cause those should probably be in their own sort of like entity, attribute value type table, like a, like a long narrow table where, you know, you link up, you know, like, you know, different, different, uh, entities to different attributes and different values than EAV table.

It’s amazing how that works out. The third thing that I look out for when I see very wide tables is, uh, clusters of missing index requests.

So like, like a lot of the times when I’ve seen crazy tables like that, um, what’s, what’s jumped out to me is that, um, missing index requests will be very specific around, which columns are in them.

So the, so like you’ll have a very similar set of search columns and a very similar set of select columns. And oftentimes you can find ways to break, to normalize that table, to break that table apart into separate tables based on what people are searching on within that table.

So you might find like, you know, setting, let’s say, like say you find 12 missing indexes and out of those 12, like four of them are selecting columns one, two, and three.

And the where it causes on columns four or five and six. And then, and the, another one, there’s like four requests and now there’s some, you know, sort of tanglement between columns eight, nine and 10 and 11, 12 and 13.

So like, you can usually find patterns within the missing index requests and within the column names, they can give you some pretty good direction and breaking. I keep saying breaking. I mean, normalizing.

It don’t mean you’re breaking the table. You’re not hurting the table. You’re not breaking your SQL Server. You’re normalizing your data, which is a wonderful thing to do. If you want more advice, if you want a lot of really good advice on doing that, Lewis Davidson has a book called relational SQL Server database design or something like that.

Let me go, let me go over to Amazon and grab a link for it. Cause it’s, I’m probably, I’m probably getting the title or something wrong.

I wish Davidson equal server design. It’s a book that I have in my bookshelf too. It’s not like I just tell people to buy these things and then screw off and let you do all the hard work.

It’s a book that I actually own and I’ve actually read. So let me stick that link in there. There you go. Wonderful.

We have a link. That’s beautiful. That’s a good thing. And let me actually, while I’m doing that, Michael, I should spell Michael, J sport 10% rule. There we go.

Swartz 10% rule. So I would send this stuff over to your developer. I would, I would get a copy of that book for your developers. And I would, I mean, unless you’re the developer, in which case I’m sorry for making money, but you know, could sell, send them, send them, show them that book, show them that link.

Say maybe, Hey, pal, we need to fix this. We need to do better. We need to do better and be better at our SQL Server, relational database design and implementation.

It’s, it’s crazy. So like, I’m like around the five year mark of consulting. I mean, obviously not independent, like, like just sort of generally like general consulting.

And I’m going to say something, uh, that is, is very, okay. I think it’s going to be annoying to a lot of people.

And as your problems are not special, your problems are very fundamental. the problems that your database is having is because you did something weird and wrong. You embraced wide tables.

You did not embrace clustered indexes. You embrace scale, our functions. You, uh, you know, did not pick a decent indexing strategy. There are so many just basic, like you, you chose data types really poorly.

Everything is a, is a long string for some reason. Everything is, uh, mistyped across tables. This, it’s very often, just very fundamental, easy fix things that like you, like should have been done from the get go, but just weren’t.

And, and it screws everything up. And it, it breaks my heart to come in and see like the same problems over and over again, where it’s just like you, like if someone had just made a couple better decisions at the outset, you could have avoided years of problems.

Years of problems. Let’s see. Kapil asks, can a query have a long running query on terabytes of database? Zero granted query memory.

I see some on large queries and don’t get why. Yes. Uh, so query memory grants. So like every query gets some memory. Because every operator.

Requires some memory to like figure out what state it’s in and what it’s doing and what it’s up to. So that every query gets some memory memory grants are very specific to a couple operations. One of them is sorts, right?

So if you sort data, you’re going to require a memory grant to do that. Uh, if you hash data, whether it’s a hash match aggregate or a hash join, you’re going to require memory to do that. And if your query requires, or if your query goes parallel, you’ll require a little extra memory to manage the exchange, the, the parallel exchanges, the buffers and the parallel exchanges.

So there are, there are three things that base, uh, in a nutshell require memory. There are some less, uh, frequent things like optimized nested loops join, which also will ask for a memory grant. And there’s, using like doing like inserts, the columnstore will also ask for a memory grant, but, uh, yeah, it’s, it’s entirely possible to have a query run across terabytes of data and not have to sort or hash anything.

It’s entirely possible for that to happen. And for a query, not to ask for a memory grant to do any of that work.

So fun stuff, huh? Like, like it’s, it might not be a great query. It might be a very slow, terrible query because at that point, I’m picturing a query with like, like lots of like little nested loops joins and, or maybe like in like index supported merge joins where, uh, memory just isn’t required to do any of that stuff.

So yeah, it’s totally possible. Uh, I find it a little suspicious that, that it’s happening on terabytes of data. Cause usually when you get terabytes of data involved, SQL Server is, uh, is pretty keen on doing some hash joins there, but who knows, who knows if you have a query plan that you can share and that you have a question about, uh, that would be a good question for DBA dot stack exchange.

Dot com. That’d be a wonderful place to ask a longer version of that, where perhaps you could provide some more detail and people could give you much more detailed answers. But I think in a nutshell, that should, that should get you going in the right direction.

Let’s see here. TZH asks, I’ve got a handful of nonclustered indexes on the clustered index key with separate columns included to cover some repeat queries. It just seemed dirty.

Yeah. I mean, you’re between a rock and a hard place, right? It’s either you, you, you got these wide tables. And if you don’t index them, uh, people are going to complain and it’s hard to index them. And if you over index them, people are going to complain.

And, uh, it all sucks. It all sucks. Uh, James says, how do I sign up to get the blogs you write sent to my email?

I added both my personal and work email to the newsletter, but never seemed to get any emails. I have checked junk mail and filters. Um, um, I don’t know what would, what would be going wrong there.

I, uh, that email list seems to work because, um, uh, what do you call it? I get, like, I see the email comes to me and I get bounce backs from everyone.

Who’s like, I get everyone’s out of office reply. Cause I’m an idiot and I don’t know how to change that in MailChimp. And when I think about Googling it, I find something better to do.

So I, but it’s, it’s kind of nice to see it’s working. It’s like, it’s like my way of knowing that I’m not just like alone in my office for eternity. Uh, but yeah, uh, I’m not sure if you shoot, if you send me an email, like if you use the contact form on my site to send me an email, um, I can, I can, I can look at your email address and look in MailChimp and kind of see what’s happening there.

It’s entirely possible that like, for some reason, I don’t know, maybe you spelled your email address wrong or, uh, or, or maybe you’re like, whatever your, your email host is just, just hates stuff from MailChimp so badly that it doesn’t even make it to spam, but just gets like immediately quarantined and junked.

It’s also entirely possible that I’ve been blacklisted again, because I don’t know. It’s happened before. I got an SSL certificate and I thought things were cool and people are still like, yeah, we don’t trust you.

I’m like, I got a certificate. I paid GoDaddy like 125 bucks for this damn certificate. How can I be untrustworthy? How, how is, how tell me how that works? Very annoying.

I don’t know. Anyway, Forrest, my furry friend. Oh, by the way, I meant to say if, if you sent me your address last week to get stickers mailed out to you, stickers were put in the mail, uh, this week.

So you should be getting them eventually. I don’t know when. Be honest with you. I wish I could, I wish I could predict and control the mail. Unfortunately, that is where my powers run out.

Anyway, Forrest says, or asks, do you ever find that knowing about memory management internals is helpful? It’s helpful when I create very specific problems that deal with memory management.

I would say that for the general public who have a pretty well-defined set of SQL Server issues, memory management is almost never, um, the issue.

The issue is that SQL Server is managing like 12 gigs of memory when it needs like 128 gigs of memory. Uh, so, you know, it’s, it’s good stuff to know.

It’s good. Like, like SQL jeopardy stuff to know. There’s a lot of good SQL jeopardy stuff to know, but you got to keep stuff, that stuff like back here, the stuff you got to keep up here is way different. Um, so yeah, it’s, it’s good to know.

It’s nice to know about, but, uh, whether it’s ever, whether I’ve ever like walked into a customer site and been like, uh, ha, I see you’ve got stolen pages. Let’s solve that.

Let’s crack that caper. And it’s been like some weird problem. It’s never been a weird problem. It’s always been like, well, you, you, Oh, you put like 500 gigs of, of memory inside SQL Server, but you forgot to raise max server memory from 64 gigs.

When you made the change or like, you’ve got a SQL Server, you are like, you’ve got a VM host with like eight SQL servers on it. And they’re all just fighting over memory constantly.

Or like, just like, like, like stuff has never been like, like, Oh, SQL services bad managing memory. It’s always been like, Oh no, there’s a people problem.

We have a big people problem here. So there’s that. Kapil says, perfect. I got it. Yes, indeed. It’s one heck of a crappy query. Too many inner joins with no sorting in a serial plan.

Woo. We, so wow. Serial plan. So we have a serial plan with no sorting over terabytes. Is there a scalar valued function involved here or a table variable involved here? Because there, there is something amok when we have a query that looks like that over that much data.

And, and there’s like no memory grant or sorting or anything that something stinks about that. Something is stinky in there.

Something is very, very stinky in there. Kapil falls. Are you planning to release some training videos? I learned a lot from, you’re available at Brent’s query tuning classes.

Yes, I am. So I’m in, I’m in a weird, weird position. And I wrote about my weird position a little bit in, in my, in my six months of consulting on my own blog posts, which will be out.

I don’t know, but I think on the anniversary of, on the anniversary of me getting laid off. So it’ll be like June 3rd, which is exactly six months from January 3rd, which is when I got beheaded.

Or when I, when I, when I had a fall from grace and became unfamous or whatever, whatever you want to call it. But yeah, so it’s, it’s been a tough mix of, you know, trying to write good training material.

In other words, like from scratch. So it’s like, I, I do like, well, I’m not, I will never ever be accused of being accused of being a perfectionist. I, I do like to provide high quality or as high quality as I’m capable of material with, with the training.

So I do try to like, you know, have, have nice pictures and have things be laid out clearly and look nice. And, and the other thing is that I don’t want to just repeat myself. So I wouldn’t want to just like completely rehash training videos that I had recorded that are up on Brent’s site.

If I’m going to, if I’m going to do this thing, it’s got to be me. So it’s, it’s tough because now I have to sort of find new ways to say things or new ways to present things.

Uh, so it is, it is writing all the material from scratch and, and it’s, it’s been going a bit more slowly than, than I’d like just because, you know, I mean, it’s a great problem to have because I, I have enough like consulting work coming in where it takes serious chunk, serious bites out of my, my time during the week where I can’t sit and dedicate it to, to writing the training, but it’s getting written.

Um, you know, I’m going to, I’m going to take, uh, some of the material from the, the, the server tuning pre-con that I’ve been doing. And, uh, I’m going to, that’s going to be like worked into some of the more advanced material.

Uh, a lot of the more advanced, uh, query tuning stuff is written right now. My, uh, my outline is, uh, like, like beginner stuff, uh, which I’m going to call starting SQL.

And that’s going to cover, um, that’s going to like do like, uh, sort of like a, like a jump, jump right into like, this is what a query does.

This is how, why it’s fast, why it’s slow. This is indexes, uh, you know, then going deeper into what is an index, what’s it, what’s a weight, you know, what’s a query plan. So like get, like getting like from like, like, like beginner stuff, but like, you know, not like, I think you’re a dummy beginner stuff.

Like I’m going to like teach you the really important, stuff about those things. Uh, then after that, I’ll probably do like a little bit of internals, not like, not like a book of boring, like, this is a database page.

This is a slaughter a type internals, like, like the stuff that I’ve found useful over the years. Um, then we’ll do like hardware stuff and then indexes more like more advanced index stuff. And then more advanced execution plan stuff.

And then query tuning. Um, I’m also hoping to have a very special guest, maybe do some, some stuff on columnstore. But I can’t say who or when, but that would be nice if that happened too.

So yeah, that’s my, that’s my plan. And, uh, and, and Josh to, to answer your email.

No, I don’t, I don’t have your address anymore. That was in my old email account. I don’t have access to any longer. Uh, so if you, if you, if you want to send me your, your actual address, that would be actually, I didn’t, I didn’t read enough of your email.

What the hell? What is this? I don’t even know what this is. Tastings. I’m not, I’m not going to you. You’re annoying me.

Email is terrible. email is the worst thing in the world. Darren says, he’s always found my classes, very informative and entertaining. Thank you, Darren.

I appreciate it. Uh, I’m glad someone does because most of the time, uh, when, whenever I’m talking to the camera, I’ll leave my office and my wife is staring at me like there’s something terribly wrong with me. I’m glad someone out there is, is, is entertained and informed.

The things that I say in here, otherwise, otherwise I don’t, I don’t know what I do. I don’t know. If, if you, if you, if you were one or the other, I would be, I would be ecstatic.

If you were informed or entertained by me, I would be thrilled. But the fact that you’re both, wow. I don’t, I don’t even know what to say to that. Enjoy, enjoy the stickers that I say to that.

Uh, yeah, that’s the thing. I don’t, I don’t carry over well to a lot of crowds. I have, I have a very specific set of people. I, I, I, I, I, I, I, I, I, I, I, I, I, I, I, I, I, I, I, I, I, I, I, I’ll tell you that that’s probably the hardest part about consulting is, is not like the consulting poker face or like, like the business stuff or the, like the, like the landing clients part.

It’s, it’s like the, the, how can I say this to like, how can I say this to people who I’ve never met in a way where like, I’m not gonna, I’m not gonna make everyone too angry.

And I think like, like that’s probably the hardest thing is, you know, like I, I need to say things in a way that they keep me interested in it. Entertain me.

But then at the same time, like I, I know someone’s not going to be happy with it. Like someone, someone’s going to be unhappy with it. Someone is going to be mad at me or, or, or think that I’m inappropriate or something.

So, ah, whatever. It’s a good time. You know what? Uh, if, if, if I had to pick a different way to make a living, I’m not sure what it would be.

Actually, I can’t, you know what I would do? I would open up a laundromat with a bar in it and I would only have it open from like, I don’t know, like 10 AM to like 8 PM, maybe 9 PM.

Cause I don’t want to like, I don’t want to have like an all night laundromat bar thing. I like, like, like, like, that’s like, like, like just washing machines full of pee would vomit would be the thing there. So I would open up a laundromat with a bar and I have very limited hours and I would just sit there and watch people do laundry.

That’s what I would do. Maybe I’d make a friendly conversation. I’d be the bartender, I guess.

I have a very limited drink menu. Cause I’m not very good at mixing things. Beer. The world doesn’t have enough shot in beer bars.

Everything’s a, everything’s a mixology bar. Everything’s going to have muddled, whatever in it. Shaved ice and rinds of things shot in a beer. Never did anyone wrong.

Anyway, uh, we’re at a, we’re at about a half hour here. And, um, uh, uh, uh, oh boy, this is a question. Hang on.

I just went through one of Joe’s blogs regarding soft Numa starting CWL server 2016. Per your experience, have you turned that feature off due to performance issues as raised by Joe in that blog? You should leave a comment on, on Joe’s blog post and ask him about it.

Cause Joe is by far a much, a much bigger expert on that Numa stuff than I am. So leave a comment with your question and, uh, mate, mate, and Joe will get back to you because he’s contractually obligated to is my guest.

No, I’m kidding. He’s not, he might ignore you, but if he does, I’m sorry. I’m sorry. I can’t control Joe. I was at first.

I was like, you know, if I’m going to do this, I should like, you know, say, okay, I’ll schedule your post to go out. And then I really just give Joe, let Joe off his leash, let Joe off his chain and do his Joe thing. Be ridiculous.

Trying to contain that man. Anyway, uh, that’s about a half hour. Uh, I’m going to, uh, I’m going to get going and, uh, get back to work or whatever you want to call it. Uh, thanks for joining me.

Uh, and I will see you most likely next week, unless something terrible happens. I’m kidding. Nothing. Unless I win the lottery, which would be wonderful. But if I do, if I win the lottery, I’m going to do this, uh, really drunk and tell you all what I actually think about you.

Goodbye. Have a great, great long weekend. if you are memorializing anything. Goodbye. Bye.

Bye. Bye. Bye. Thank you.

Video Summary

In this video, I delve into some fundamental SQL Server and database design principles. Starting off, I discuss the importance of avoiding wide tables—tables with more than 100 columns—and how they can lead to indexing challenges and maintenance headaches. I also share a few patterns to look out for in your schema that might indicate poorly designed tables, such as prefixing column names or using numeric suffixes for similar data. Additionally, I explore the concept of Michael J. Swartz’s 10% rule, emphasizing that while SQL Server supports up to 1,024 columns per table, it’s wise to keep your tables narrow and well-structured.

Moving on, I cover memory grants in queries, explaining how they are necessary for operations like sorting and hashing but not always required. I provide practical examples of scenarios where a query can process terabytes of data without needing any memory grant, highlighting the importance of understanding these nuances for optimizing performance. Throughout the video, I offer actionable advice on improving your database design, referencing Lewis Davidson’s book “Relational Database Design for SQL Server” as an excellent resource to guide developers in creating more efficient and maintainable databases.

Full Transcript

So, let’s see. Science and technology is the, I don’t know, channel, stream that this thing takes place in. I don’t know if that’s really accurate. I don’t know if I’d call it science or technology. Maybe, maybe neither. It’s not terribly scientific. And aside from the fact that I use a computer to broadcast, I have absolutely no technology in here. I did get a new phone. Got a Pixel 3 XL. Pretty psyched about that. It’s a picture of my kid picking her nose on there. She’s a good time. So, just to kill a little time until questions come in, I see people in here, which is very exciting. Thank you for joining me. I’ve been writing some blog posts about my first six months of consulting independently. And, I don’t know, it was a real tear-jerker. There were some emotional moments in there. I’m kidding.

I replaced feelings with whiskey many decades ago. But, anyway, I’ve been, shut up, motorcycle. Back to your biker gang. I’ve been getting allergy shots for, I want to say, like, three years. At least three years. I’m on, like, the monthly maintenance shots now where, like, I just, like, they don’t even make me safe for a half hour anymore. I get my shot and I hang out for, like, 10, 15 and just say goodbye. Whiskey. Whiskey. Tisigr asks, whiskey or bourbon? I would never drink bourbon. Bourbon is just hyped up maple syrup. I would never drink bourbon. I’m a scotch guy.

And, like, very specifically, I like Iowa scotches. I like the stuff that tastes like burning band-aids. That’s my jam. But, yeah, we’ve been getting allergy shots for three years now, monthly maintenance. And the last, I don’t know what changed in the, I don’t know what changed in the world. I don’t know what new life form showed up on planet Earth. But it’s, like, I’ve never had a shot in my life. For, like, a long time, they were great.

I, like, I went from having, like, severe, like, eyes running, nose constantly bubbling, gross stuff, allergies, like, unable to function allergies, to, like, I would take maybe, like, antihistamine, like, once a week or so or, like, once every couple weeks. It was great. But, man, the last week has been absolutely positively brutal. Like, I’ve been taking stuff every day. Like, every morning, I wake up at, like, 3, 4 in the morning with, like, my face in awful condition.

I don’t know what the hell is different this year. But, man, it is bad. Bad. Ugh. It’s terrible.

Anyway, I don’t know. I don’t know. Someone ask a question. There are at least six of you here.

I don’t know how many more are going to come in. There are some of you. Someone has to have a SQL Server question. You can’t just come here to hear me complain about things or blab on and on about blog posts and whatnot.

Someone has to have a question about SQL Server. Please, God, someone have a question. Come on, there’s, like, 10 of you. Laura says, do you have any thoughts on replacing temp tables with in-memory OLTP?

Yeah, don’t. Hecaton is like herpes. It doesn’t have the nerve to kill you.

It just hangs around being awful for the rest of your life and flaring up at inappropriate times. So, I just have not found anything compelling about Hecaton. It seems like every time…

And this is not just me. This is some very smart people that I’m friends with. Every time they think that they have a specific problem with latches that Hecaton might solve, it is an absolute dead end.

Absolute dead end. In SQL Server 2019, TempDB is going to use… Well, you can use TempDB as, like, an Hecaton-y thing anyway.

I would pry this whole lot to see if that happens. But, whereas, I guess the question is, why? Why?

What are you trying to fix? What is a problem that we’re trying to solve here? What do you think Hecaton is going to make better in your stored procedure that you want to use in memory for?

That’s the big question. What’s going to get better about it? What’s going to get better? I don’t know.

Do, do. Do, do. Do, do. Try to have better indexing of… What on earth? You can index TempTables now.

What’s missing from your current TempTable indexing that Hecaton is going to provide a safe and secure solution for? That’s what I’m curious about.

Too slow. Well, so here’s the thing. What’s too slow? Creating the index or populating the Temp…

or populating the table? Josh says Hecaton indexes are extra confusing. Yeah, they are. So who knows how many hash buckets you should set up? Creating the index.

So… Creating the index is too slow. Okay. Fair enough. See, this is one of those things where it’s like, we’re going to end up going down a real rabbit hole.

Because I’m going to ask you what kind of index. And what kind of data types you’re indexing. And how many rows it is. And a lot of other stuff.

And this is… I hope you’re prepared. Because this can go on for a long, long time. I know.

Then you’d have to actually look at code. Bad news is you’d actually have to look at code to implement Hecaton. and get it set up and running there. If you don’t…

If creating an index is slow now on a Temp table, I don’t think it’s going to be any faster in Hecaton. It’s not free.

Nothing is free. Nothing is free. Yeah, no. Well, you know, my…

So Hecaton is very specifically designed to deal with locking and lashing issues. The in-memory portion, I think, is…

The misgiving around it is that it’ll make any workload faster. And that’s not really true. And the use cases where I’ve seen Hecaton be successful is with really large-scale, fast data ingestion into tables where data is not going to live for very long.

So the example that I always give, because it’s the example that I’ve seen work best, was with online gambling, where data had to come in very fast. And we cared about that data for a very short amount of time, where we wanted very fast updates and being able to get to that data to be snappy and not get blocked up and locked.

And then after like an hour or so, or whenever the betting thing is over, they get pushed out to regular on this table. So that’s the only time.

It’s the only time I’ve ever seen Hecaton be successful. For every other weird niche thing that someone’s like, oh, I bet if we just did this in memory, it’d be faster.

It has never worked out. Never worked out. I don’t know. It’s like, people hear about these features and they, I don’t know, the pamphlets that Microsoft comes up with for these things are amazing because they make it seem like they’re going to fix every single problem that you’re having.

They just sound like this, like golden acres retirement home for your data. And geez, never, never seems to go.

Yeah. So like with the Lars, I guess to put it, put it as succinctly as possible. you can try it, but be very careful with the database that you try it in because once you have created an in-memory file group in a database, one cannot drop an in-memory file group.

One must drop and recreate the database. So if you’re going to try it somewhere, create a new database and enable things there. Don’t do it with a database you care about.

So that’s about all I got. Let’s see. TZH X says, do you have a rule of thumb for how wide a table is too wide? Currently have a bunch of core tables in the system with a hundred plus columns each.

Yeah. You’re about there. So I think one way to, one way to phrase this really well is the maximum number of columns you can have in a table is a thousand and 24.

And a really smart friend of mine, Canadian fella, says it has, has sort of a, what do you call is it? Michael’s Michael J. Swartz 10% rule with, if you’re using 10% of a maximum of a limit, right?

Cause these are limits. They are not goals. They’re not things that you are trying to attain. They are things that they are, they are like, like the, the capacity at which SQL Server will stop functioning. So you’re, you’re right about there.

And the thing that really sucks about a hundred column tables is they’re really, really difficult to index. Well, unless you don’t care about read speeds from them, unless you’re just dumping data in there and not really doing anything with it, then it becomes really hard to index that.

Cause people are, people are going to you know, they’re going to want to search on, on weird sets of columns. And they’re going to want to select weird sets of columns and potentially order by weird sets of columns.

And all that adds up to is a lot of a heartache and pain and trying to index those things. So when I start seeing tables like that, that there are three patterns that I look out for very specifically.

one is column names that have prefixes. So like customer name, customer number, customer address, customer, things like that. Cause those should all be in a table called customer. The other thing that I look out for is, uh, columns that end in numbers.

So like phone one, phone two, phone three, email one, email two, email three. I’m also, I also pay attention to those. Cause those should probably be in their own sort of like entity, attribute value type table, like a, like a long narrow table where, you know, you link up, you know, like, you know, different, different, uh, entities to different attributes and different values than EAV table.

It’s amazing how that works out. The third thing that I look out for when I see very wide tables is, uh, clusters of missing index requests.

So like, like a lot of the times when I’ve seen crazy tables like that, um, what’s, what’s jumped out to me is that, um, missing index requests will be very specific around, which columns are in them.

So the, so like you’ll have a very similar set of search columns and a very similar set of select columns. And oftentimes you can find ways to break, to normalize that table, to break that table apart into separate tables based on what people are searching on within that table.

So you might find like, you know, setting, let’s say, like say you find 12 missing indexes and out of those 12, like four of them are selecting columns one, two, and three.

And the where it causes on columns four or five and six. And then, and the, another one, there’s like four requests and now there’s some, you know, sort of tanglement between columns eight, nine and 10 and 11, 12 and 13.

So like, you can usually find patterns within the missing index requests and within the column names, they can give you some pretty good direction and breaking. I keep saying breaking. I mean, normalizing.

It don’t mean you’re breaking the table. You’re not hurting the table. You’re not breaking your SQL Server. You’re normalizing your data, which is a wonderful thing to do. If you want more advice, if you want a lot of really good advice on doing that, Lewis Davidson has a book called relational SQL Server database design or something like that.

Let me go, let me go over to Amazon and grab a link for it. Cause it’s, I’m probably, I’m probably getting the title or something wrong.

I wish Davidson equal server design. It’s a book that I have in my bookshelf too. It’s not like I just tell people to buy these things and then screw off and let you do all the hard work.

It’s a book that I actually own and I’ve actually read. So let me stick that link in there. There you go. Wonderful.

We have a link. That’s beautiful. That’s a good thing. And let me actually, while I’m doing that, Michael, I should spell Michael, J sport 10% rule. There we go.

Swartz 10% rule. So I would send this stuff over to your developer. I would, I would get a copy of that book for your developers. And I would, I mean, unless you’re the developer, in which case I’m sorry for making money, but you know, could sell, send them, send them, show them that book, show them that link.

Say maybe, Hey, pal, we need to fix this. We need to do better. We need to do better and be better at our SQL Server, relational database design and implementation.

It’s, it’s crazy. So like, I’m like around the five year mark of consulting. I mean, obviously not independent, like, like just sort of generally like general consulting.

And I’m going to say something, uh, that is, is very, okay. I think it’s going to be annoying to a lot of people.

And as your problems are not special, your problems are very fundamental. the problems that your database is having is because you did something weird and wrong. You embraced wide tables.

You did not embrace clustered indexes. You embrace scale, our functions. You, uh, you know, did not pick a decent indexing strategy. There are so many just basic, like you, you chose data types really poorly.

Everything is a, is a long string for some reason. Everything is, uh, mistyped across tables. This, it’s very often, just very fundamental, easy fix things that like you, like should have been done from the get go, but just weren’t.

And, and it screws everything up. And it, it breaks my heart to come in and see like the same problems over and over again, where it’s just like you, like if someone had just made a couple better decisions at the outset, you could have avoided years of problems.

Years of problems. Let’s see. Kapil asks, can a query have a long running query on terabytes of database? Zero granted query memory.

I see some on large queries and don’t get why. Yes. Uh, so query memory grants. So like every query gets some memory. Because every operator.

Requires some memory to like figure out what state it’s in and what it’s doing and what it’s up to. So that every query gets some memory memory grants are very specific to a couple operations. One of them is sorts, right?

So if you sort data, you’re going to require a memory grant to do that. Uh, if you hash data, whether it’s a hash match aggregate or a hash join, you’re going to require memory to do that. And if your query requires, or if your query goes parallel, you’ll require a little extra memory to manage the exchange, the, the parallel exchanges, the buffers and the parallel exchanges.

So there are, there are three things that base, uh, in a nutshell require memory. There are some less, uh, frequent things like optimized nested loops join, which also will ask for a memory grant. And there’s, using like doing like inserts, the columnstore will also ask for a memory grant, but, uh, yeah, it’s, it’s entirely possible to have a query run across terabytes of data and not have to sort or hash anything.

It’s entirely possible for that to happen. And for a query, not to ask for a memory grant to do any of that work.

So fun stuff, huh? Like, like it’s, it might not be a great query. It might be a very slow, terrible query because at that point, I’m picturing a query with like, like lots of like little nested loops joins and, or maybe like in like index supported merge joins where, uh, memory just isn’t required to do any of that stuff.

So yeah, it’s totally possible. Uh, I find it a little suspicious that, that it’s happening on terabytes of data. Cause usually when you get terabytes of data involved, SQL Server is, uh, is pretty keen on doing some hash joins there, but who knows, who knows if you have a query plan that you can share and that you have a question about, uh, that would be a good question for DBA dot stack exchange.

Dot com. That’d be a wonderful place to ask a longer version of that, where perhaps you could provide some more detail and people could give you much more detailed answers. But I think in a nutshell, that should, that should get you going in the right direction.

Let’s see here. TZH asks, I’ve got a handful of nonclustered indexes on the clustered index key with separate columns included to cover some repeat queries. It just seemed dirty.

Yeah. I mean, you’re between a rock and a hard place, right? It’s either you, you, you got these wide tables. And if you don’t index them, uh, people are going to complain and it’s hard to index them. And if you over index them, people are going to complain.

And, uh, it all sucks. It all sucks. Uh, James says, how do I sign up to get the blogs you write sent to my email?

I added both my personal and work email to the newsletter, but never seemed to get any emails. I have checked junk mail and filters. Um, um, I don’t know what would, what would be going wrong there.

I, uh, that email list seems to work because, um, uh, what do you call it? I get, like, I see the email comes to me and I get bounce backs from everyone.

Who’s like, I get everyone’s out of office reply. Cause I’m an idiot and I don’t know how to change that in MailChimp. And when I think about Googling it, I find something better to do.

So I, but it’s, it’s kind of nice to see it’s working. It’s like, it’s like my way of knowing that I’m not just like alone in my office for eternity. Uh, but yeah, uh, I’m not sure if you shoot, if you send me an email, like if you use the contact form on my site to send me an email, um, I can, I can, I can look at your email address and look in MailChimp and kind of see what’s happening there.

It’s entirely possible that like, for some reason, I don’t know, maybe you spelled your email address wrong or, uh, or, or maybe you’re like, whatever your, your email host is just, just hates stuff from MailChimp so badly that it doesn’t even make it to spam, but just gets like immediately quarantined and junked.

It’s also entirely possible that I’ve been blacklisted again, because I don’t know. It’s happened before. I got an SSL certificate and I thought things were cool and people are still like, yeah, we don’t trust you.

I’m like, I got a certificate. I paid GoDaddy like 125 bucks for this damn certificate. How can I be untrustworthy? How, how is, how tell me how that works? Very annoying.

I don’t know. Anyway, Forrest, my furry friend. Oh, by the way, I meant to say if, if you sent me your address last week to get stickers mailed out to you, stickers were put in the mail, uh, this week.

So you should be getting them eventually. I don’t know when. Be honest with you. I wish I could, I wish I could predict and control the mail. Unfortunately, that is where my powers run out.

Anyway, Forrest says, or asks, do you ever find that knowing about memory management internals is helpful? It’s helpful when I create very specific problems that deal with memory management.

I would say that for the general public who have a pretty well-defined set of SQL Server issues, memory management is almost never, um, the issue.

The issue is that SQL Server is managing like 12 gigs of memory when it needs like 128 gigs of memory. Uh, so, you know, it’s, it’s good stuff to know.

It’s good. Like, like SQL jeopardy stuff to know. There’s a lot of good SQL jeopardy stuff to know, but you got to keep stuff, that stuff like back here, the stuff you got to keep up here is way different. Um, so yeah, it’s, it’s good to know.

It’s nice to know about, but, uh, whether it’s ever, whether I’ve ever like walked into a customer site and been like, uh, ha, I see you’ve got stolen pages. Let’s solve that.

Let’s crack that caper. And it’s been like some weird problem. It’s never been a weird problem. It’s always been like, well, you, you, Oh, you put like 500 gigs of, of memory inside SQL Server, but you forgot to raise max server memory from 64 gigs.

When you made the change or like, you’ve got a SQL Server, you are like, you’ve got a VM host with like eight SQL servers on it. And they’re all just fighting over memory constantly.

Or like, just like, like, like stuff has never been like, like, Oh, SQL services bad managing memory. It’s always been like, Oh no, there’s a people problem.

We have a big people problem here. So there’s that. Kapil says, perfect. I got it. Yes, indeed. It’s one heck of a crappy query. Too many inner joins with no sorting in a serial plan.

Woo. We, so wow. Serial plan. So we have a serial plan with no sorting over terabytes. Is there a scalar valued function involved here or a table variable involved here? Because there, there is something amok when we have a query that looks like that over that much data.

And, and there’s like no memory grant or sorting or anything that something stinks about that. Something is stinky in there.

Something is very, very stinky in there. Kapil falls. Are you planning to release some training videos? I learned a lot from, you’re available at Brent’s query tuning classes.

Yes, I am. So I’m in, I’m in a weird, weird position. And I wrote about my weird position a little bit in, in my, in my six months of consulting on my own blog posts, which will be out.

I don’t know, but I think on the anniversary of, on the anniversary of me getting laid off. So it’ll be like June 3rd, which is exactly six months from January 3rd, which is when I got beheaded.

Or when I, when I, when I had a fall from grace and became unfamous or whatever, whatever you want to call it. But yeah, so it’s, it’s been a tough mix of, you know, trying to write good training material.

In other words, like from scratch. So it’s like, I, I do like, well, I’m not, I will never ever be accused of being accused of being a perfectionist. I, I do like to provide high quality or as high quality as I’m capable of material with, with the training.

So I do try to like, you know, have, have nice pictures and have things be laid out clearly and look nice. And, and the other thing is that I don’t want to just repeat myself. So I wouldn’t want to just like completely rehash training videos that I had recorded that are up on Brent’s site.

If I’m going to, if I’m going to do this thing, it’s got to be me. So it’s, it’s tough because now I have to sort of find new ways to say things or new ways to present things.

Uh, so it is, it is writing all the material from scratch and, and it’s, it’s been going a bit more slowly than, than I’d like just because, you know, I mean, it’s a great problem to have because I, I have enough like consulting work coming in where it takes serious chunk, serious bites out of my, my time during the week where I can’t sit and dedicate it to, to writing the training, but it’s getting written.

Um, you know, I’m going to, I’m going to take, uh, some of the material from the, the, the server tuning pre-con that I’ve been doing. And, uh, I’m going to, that’s going to be like worked into some of the more advanced material.

Uh, a lot of the more advanced, uh, query tuning stuff is written right now. My, uh, my outline is, uh, like, like beginner stuff, uh, which I’m going to call starting SQL.

And that’s going to cover, um, that’s going to like do like, uh, sort of like a, like a jump, jump right into like, this is what a query does.

This is how, why it’s fast, why it’s slow. This is indexes, uh, you know, then going deeper into what is an index, what’s it, what’s a weight, you know, what’s a query plan. So like get, like getting like from like, like, like beginner stuff, but like, you know, not like, I think you’re a dummy beginner stuff.

Like I’m going to like teach you the really important, stuff about those things. Uh, then after that, I’ll probably do like a little bit of internals, not like, not like a book of boring, like, this is a database page.

This is a slaughter a type internals, like, like the stuff that I’ve found useful over the years. Um, then we’ll do like hardware stuff and then indexes more like more advanced index stuff. And then more advanced execution plan stuff.

And then query tuning. Um, I’m also hoping to have a very special guest, maybe do some, some stuff on columnstore. But I can’t say who or when, but that would be nice if that happened too.

So yeah, that’s my, that’s my plan. And, uh, and, and Josh to, to answer your email.

No, I don’t, I don’t have your address anymore. That was in my old email account. I don’t have access to any longer. Uh, so if you, if you, if you want to send me your, your actual address, that would be actually, I didn’t, I didn’t read enough of your email.

What the hell? What is this? I don’t even know what this is. Tastings. I’m not, I’m not going to you. You’re annoying me.

Email is terrible. email is the worst thing in the world. Darren says, he’s always found my classes, very informative and entertaining. Thank you, Darren.

I appreciate it. Uh, I’m glad someone does because most of the time, uh, when, whenever I’m talking to the camera, I’ll leave my office and my wife is staring at me like there’s something terribly wrong with me. I’m glad someone out there is, is, is entertained and informed.

The things that I say in here, otherwise, otherwise I don’t, I don’t know what I do. I don’t know. If, if you, if you, if you were one or the other, I would be, I would be ecstatic.

If you were informed or entertained by me, I would be thrilled. But the fact that you’re both, wow. I don’t, I don’t even know what to say to that. Enjoy, enjoy the stickers that I say to that.

Uh, yeah, that’s the thing. I don’t, I don’t carry over well to a lot of crowds. I have, I have a very specific set of people. I, I, I, I, I, I, I, I, I, I, I, I, I, I, I, I, I, I, I, I, I, I, I, I’ll tell you that that’s probably the hardest part about consulting is, is not like the consulting poker face or like, like the business stuff or the, like the, like the landing clients part.

It’s, it’s like the, the, how can I say this to like, how can I say this to people who I’ve never met in a way where like, I’m not gonna, I’m not gonna make everyone too angry.

And I think like, like that’s probably the hardest thing is, you know, like I, I need to say things in a way that they keep me interested in it. Entertain me.

But then at the same time, like I, I know someone’s not going to be happy with it. Like someone, someone’s going to be unhappy with it. Someone is going to be mad at me or, or, or think that I’m inappropriate or something.

So, ah, whatever. It’s a good time. You know what? Uh, if, if, if I had to pick a different way to make a living, I’m not sure what it would be.

Actually, I can’t, you know what I would do? I would open up a laundromat with a bar in it and I would only have it open from like, I don’t know, like 10 AM to like 8 PM, maybe 9 PM.

Cause I don’t want to like, I don’t want to have like an all night laundromat bar thing. I like, like, like, like, that’s like, like, like just washing machines full of pee would vomit would be the thing there. So I would open up a laundromat with a bar and I have very limited hours and I would just sit there and watch people do laundry.

That’s what I would do. Maybe I’d make a friendly conversation. I’d be the bartender, I guess.

I have a very limited drink menu. Cause I’m not very good at mixing things. Beer. The world doesn’t have enough shot in beer bars.

Everything’s a, everything’s a mixology bar. Everything’s going to have muddled, whatever in it. Shaved ice and rinds of things shot in a beer. Never did anyone wrong.

Anyway, uh, we’re at a, we’re at about a half hour here. And, um, uh, uh, uh, oh boy, this is a question. Hang on.

I just went through one of Joe’s blogs regarding soft Numa starting CWL server 2016. Per your experience, have you turned that feature off due to performance issues as raised by Joe in that blog? You should leave a comment on, on Joe’s blog post and ask him about it.

Cause Joe is by far a much, a much bigger expert on that Numa stuff than I am. So leave a comment with your question and, uh, mate, mate, and Joe will get back to you because he’s contractually obligated to is my guest.

No, I’m kidding. He’s not, he might ignore you, but if he does, I’m sorry. I’m sorry. I can’t control Joe. I was at first.

I was like, you know, if I’m going to do this, I should like, you know, say, okay, I’ll schedule your post to go out. And then I really just give Joe, let Joe off his leash, let Joe off his chain and do his Joe thing. Be ridiculous.

Trying to contain that man. Anyway, uh, that’s about a half hour. Uh, I’m going to, uh, I’m going to get going and, uh, get back to work or whatever you want to call it. Uh, thanks for joining me.

Uh, and I will see you most likely next week, unless something terrible happens. I’m kidding. Nothing. Unless I win the lottery, which would be wonderful. But if I do, if I win the lottery, I’m going to do this, uh, really drunk and tell you all what I actually think about you.

Goodbye. Have a great, great long weekend. if you are memorializing anything. Goodbye. Bye.

Bye. Bye. Bye. Thank you.

Going Further


If this is the kind of SQL Server stuff you love learning about, you’ll love my training. Blog readers get 25% off the Everything Bundle — over 100 hours of performance tuning content. Need hands-on help? I offer consulting engagements from targeted investigations to ongoing retainers. Want a quick sanity check before committing to a full engagement? Schedule a call — no commitment required.

Last Week’s Almost Definitely Not Office Hours: May 3

ICYMI


Last week’s thrilling, stunning, flawless episode of whatever-you-wanna-call-it.

Video Summary

In this video, I discuss various technical topics related to SQL Server and database optimization. Starting off, I address issues around bad estimates and locking, explaining how local variables can lead to inaccurate row estimation and resulting in page-level locks instead of the more efficient row-level locks when using a recompile hint or correct parameters. Moving on, I delve into the problem of parameter sniffing, proposing an approach where SQL Server could cache multiple plans based on histogram data to handle different levels of data skew. This would involve analyzing histograms for potential skew and creating candidate plans that can be selected at runtime based on the actual parameter values. Lastly, I touch upon scaling this solution across multiple histograms and consider how query store can help mitigate some of these issues by automatically caching and reusing efficient execution plans.

Full Transcript

Yeah. So that’s, that’s apparently live. Fun. Fun. I feel like I don’t expect anyone to show up. I haven’t been here for two weeks. I’ve abandoned you. I’ve left you. Nice, sweet people, completely alone. Not here answering your questions. Uh, I was even a little bit late today because I had to, uh, do the, an adult thing. And I had to talk to an accountant about taxes. I was, I was like, I don’t know how to do anything. He’s just like, what do you do for work? I’m like, I’m a database consultant. He’s like, that sounds smart. And it was like, it’s like, yeah, yeah, but I just, I just don’t know anything about money or taxes. And he just, like, it’s, it’s funny when, um, people, people think that, uh, you know, when you’re, you’re smart about one thing that should have carry over to other things. Not the case. Not the case. I am, I am useless and hopeless when it comes to other, other endeavors.

All right. So I have, I have to Google some tax forms and I have to go to some websites about dealing with stuff. And it’s just not, not enjoyable. Not enjoyable at all. Anyway, there are, there are exactly two of you here, which is exactly how many people I would expect to show up after two weeks of not.

So if you have a question, you can, you can ask a question. My video just showed up. That’s weird. Uh, I had a weird error before this thing even started. Uh, Jesus Christ.

Who’s going to hear? Uh, before we even started a bump, uh, when I went to start up the webcast, they told me that my webcam was in use and I had to unplug it and plug it back in, but then video showed up immediately.

Expatriate never paid. I would love to, I would love to, uh, I don’t know, live somewhere in Europe. My, my new retirement plan is smoking cigarettes in a French graveyard. That’s, that’s how I’m going to spend my, my waning years.

Getting, getting ready to die. Surrounded by death and a cloud of cigarette smoke. Assuming they just still allowed to smoke by then. I don’t, I don’t know if I’m going to be able to retire until I’m very old. So.

There’s that to look forward to. I hope anyway. All right. Taking a look around here. I’m, uh, I’m getting some blog posts scheduled for the next few weeks. Cause, uh, all the travel and other nonsense really put me behind my blogging schedule.

I wasn’t, uh, I wasn’t, I wasn’t, uh, I wasn’t in my game with that. Getting caught up is tough because then I have to think about stuff very quickly. Oh, whatever.

All right. Six of you. One of you has to have a question about something. SQL Server. I don’t know. What else do you people do? What do you do for fun? I need a haircut.

This wave is annoying me. I’m going to just mess my hair up so I don’t have to look at it. There we go. I feel better now. It’s getting aggravating. Seven of you.

Seven of you and no SQL Server questions. Ah, man. It’s a downer. I was in Minneapolis, Minnesota, visiting a client on site, uh, without, without revealing too much.

It was a large, uh, what do you call it? God, why can’t I think of the name now? Um, CPA accounting. It’s a large accounting company.

And, uh, it was very funny to have me walking around there looking the way I look with all these people who have clearly done all the traditionally correct things in life just staring at me. And this, why is anyone listening?

Why is anyone talking to him? Like, huh? It was, it was, it was amusing. Anyway, we have some questions. Some questions. Uh, let’s see. Uh, do we have any recommendations on where to start if you want to enable columnstore indexes? Well, yeah.

First is to figure out if it’s even good for what you’re doing. If you’re doing OLTP, you probably don’t need columnstore. And that’s just kind of the end of it. If you are, if you have a data warehouse or you have some reporting needs, then columnstore might be a good idea.

Uh, but further, further that depends on which, version of SQL Server you’re on. If you’re on 2012, it’s not a good. Comptor is not your friend on 2012, 2014.

It might kind of be your friend. 2016. You’ll be on okay terms. 2017 clustered columnstore is probably going to be your jam. Yeah. So 2006. So I would, I would, you know, uh, you could try cluster, clustered columnstore out.

I promise I can talk English. Uh, so, you know, it, it, take a, take a look at, so a kind of a fun way to see if batch mode would be helpful at all is to just create like an empty temp table with a clustered columnstore index on it and do like that, that left join on one equals zero to, uh, to one of your data warehouse queries and see if you get batch mode operations and see if it improves the speed at all.

So there’s a good way to figure out if it’s good, if it will help at all is to do that and get batch mode involved on some level. Uh, just so, you know, you can be like, well, you know, like rather than going through all the pain of converting, because like there, there are some limitations with columnstore, especially in 2016.

temp tables and see if batch mode helps anything. Peter says that he tweeted a question after the last office hours. That was like three weeks ago. Now, man, three, three, three damn weeks ago.

I can’t remember three days ago. Three hours ago was fuzzy. So the forest says, what are some common misconceptions you encounter among people new to SQL Server? God, like everything. It depends. Like, I don’t know, like depending on how new they are, they might not have any conceptions.

It could be misconceptions. So the fact that like a lot of the times, um, you know, people just think it’s like an Excel file, right? You know, like what they see in management studio is the way data is stored. It’s like, this will be easy.

I can do whatever I want. Uh, but some, I don’t know, common misconceptions. Jeez Louise. People think it’s a lot smarter than it is so that it’ll take care of a lot more things than it actually will. Databases are not your friend. They do not have your back.

Databases want you to fail. Uh, Louie says, what version of SQL was the client on? Uh, I believe, if I remember correctly. Yeah, it was 2014. Cause there was some, there was, cause I, cause I wanted to see if query store was around, but they weren’t on 2016 yet.

Let’s see here. Peter says, given proc one with two batches, with two plans each, are there four possible proc plan combos or two, uh, four. If branching and store procedures means that you could, means that when, when, when an initial plan is compiled, all possible plans are explored and compiled.

What really sucks is that they do it with the initial set of variables. So if like any part of your branching is like, if you want to select from this table, then we’ll follow this branch.

Or if you want to join these tables and we follow a different branch and you get, and I don’t know, it’s weird, weird, bad things happen with nulls. It’s, it’s awful and terrible. And you should either use dynamic SQL or nested store procedures to do it. Cause if logic and store procedures, it’s just stinky.

Oh man. I got thinking about smoking cigarettes in a French graveyard. Now, of course, all I want to do is smoke a cigarette in a French graveyard. Damn. Uh, Max says, can you explain what you meant in bad estimates and locking?

I couldn’t get the point. What did you do to fix the issue? Uh, yeah. So, uh, my, my blog post, I think it was today. I forget when things get published. Uh, I ran into an issue recently with declared variables and declared variables were causing a bad, bad overestimate and how many rows might, uh, result from, uh, a where clause.

So what was happening is, uh, Corey was running a local variable was in use in the where clause. And, uh, as a result, many, many more rows were expected to be, be returned. So the lock granularity was at the page level rather than at the row level. But the, when you use a recompile hint or you use a correct, you know, type of parameter or literal or whatever, uh, then you’ll get the correct estimate and you’ll get key level locks.

So, yeah, that was that. He just says, hashtag SQL office hours. Let’s make it happen. Ah, boy, that sounds exhausting.

Who has time for that? Is that Twitter? Sounds awful. Twitter’s the worst. I wish, I wish that I could like just run screaming. On Twitter. Run screaming.

Speaking of which, I’m going to see if anything is in there. What are you saying here? Uh, different thing. Yeah, apparently that was. Wait, forest. What did you just like?

Did this thing, this thing publish early? Oh, it must be just a blank video. Damn. Whatever. far says if you were at Microsoft, how would you, how would you solve parameters sniffing? That’s a great question.

And, uh, I think the answer would be that you couldn’t just have a single iteration of a plan. Based on the first set of parameters that get passed in, you would need, uh, uh, you would need to be a little bit more, I guess, I guess Microsoft would call, call it intelligent about, uh, whether you pursue a plan, a parameterized plan, or whether you, uh, go looking for other possibilities.

So like, like just like taking the simplest example of, uh, store procedure with a single parameter. If you have, if you look, if, if SQL Server looks at the histogram for that parameter and notes some amount of skew across values, it should mark that plan as, uh, a candidate for, uh, additional branches.

And if it, if it, if it, if it, if it, if it doesn’t denote skew, then it should just say, no, this one, this one’s going to be pretty stable no matter what. And then if you have a candidate plan, it says automatic plan correction. No, cause I don’t, cause I don’t, I don’t want SQL Server changing its idea mid flight.

I want SQL Server caching a plan and I want SQL Server using a, a completely different query from the start. So what, what, what I would like to see happen is you, you would have like a candidate plan for, uh, for like, you know, say, you know, typical example, stack overflow database users table reputation equals two.

You look at the reputation histogram, the reputation equals one users have quite a different set of data in the table from pretty much any other reputation, like more than all the other reputations combined. Uh, so what, what you would want to do is have a plan that is sort of suitable for, uh, unskewed values.

And then a plan that is rather suitable for the skewed values. So you would have the, you take this plan, SQL Server would look at a parameter, for that plan, look at the histogram that it’s going to use for that and say, all right, well, I know that I see that reputation equals one is a doozy. There’s like three, like, I don’t know, however many users for that 3 million or so users.

And then, uh, I would look at the rest of the rest of the values and say, well, all together there, they hardly break 3 million altogether. It’s like, you know, a million and a half of them for any value other than one, we can use this plan. And then for reputation equals one, we can use this plan.

Like, like it would just, it would be a, it would be a matter of like judging candidacy based on, uh, you know, histogram data, which can be wrong, but I would much rather see the attempt made than just kind of left to kick in the wind. I don’t, I don’t want to see plan correction while a plant, while a query is in flight, because, you know, that seems hacky.

That seems tacked on to me. I would rather see SQL Server do this when a plan is first compiled to say, look, there are different branches that are going to make sense for different levels of data flow. So I think like the next value down from one is like a 10 or 11 or something.

And so it might make sense to say, hey, look, we might need a different plan for the 10s and 11s, but everyone else, or the ones and ones and 10s and 11s, but everyone else can use this kind of small plan. So I think, I think really the solution would be sort of an initial look at histograms involved and figuring out if there is skew in those histograms. And if skew is detected, then you would, you know, cache different versions of the plan just because SQL Server already has compiled time and runtime values in the plan.

So it knows there’s a difference. It’s able to sniff the runtime value and do stuff with it. And if you already had a plan available for a different, for a different parameter, you could, you wouldn’t have to worry about plan caching or you wouldn’t have to worry about compiling a new plan at runtime.

You would just, just use a different branch of the plan based on the, the runtime value. Seems easy to me. I know it’s going to be harder when there’s like, when there’s like 50 parameter store procedures, but those probably aren’t the best candidates to start trying to fix parameters sniffing for.

There’s like, you know, let’s say between one and five parameters, it might, might be doable. Granted, there’s going to be some weird math in there, but math isn’t my strong suit. So I wouldn’t be in charge of that. I would just, I would just draw the big picture and let other people fill in the details.

Yeah. It would be, it’d be like a, like a Franken plan, right? It would just be like sewing different chunks of query plans together based on, based on what’s going on.

It’d be fun. I think it’d be fun. They should hire me just to do that. I would love to do that. Just get rid of plank. You could just get rid of plank hashing. We do have query store. So we don’t, don’t need the plank hash.

We could just turn on query store and say, Hey, you match this plan. Let’s use that. Right. It’d be fun. Peter says any thoughts on scaling that to multiple histograms or is most sniffing trouble isolated to one set of stats usually? Yes and no.

I think there are some statistics that are, are generally more uniform than others. You know, date, date values tend to be a bit more uniform. Granted, you’ll have some skew. If like, you know, you’re thinking about like an OLTP thing.

If you have a big sale or like black Friday comes along, you might, you probably have some more values in there then, but I think generally if you know, you’ll have a fairly stable, you know, per day number of orders or something. So, you know, that’s one thing you might grow like year over year or month over month, but you know, that’s where people, people generally aren’t when they look at historical stuff, they might, they’re probably looking at like just the newest stuff anyway.

So kind of like less of an issue there. Um, let’s go into multiple histograms. Yeah, of course it’d be tougher, but at the same time, it’s pretty easy to, to, to judge at, at that point, if you’re using multiple histograms, which of those histograms have skew involved in them.

So it would, it would purely be, uh, for histograms that show skew at compile time. It wouldn’t be like, you know, every, every history, every histogram involved. It’d be like your histogram would have to show, like, you know, some amount of skew towards one value or a group of values in order to make sense.

And for that to make sense, if they got rid of the plan cash, Josh, what, what would, how, what would I do with my, my time? We’re not writing XML queries. What would I do with myself?

What would I look at and stare and point and shock and horror? What would I do? What would I do? There is not coffee enough in the world today. Work out more.

That’d be nice. I mean, blog less, work out more. Ooh, Darren’s, Darren’s firing some shots. He says, obviously you don’t run high enough batch requests to second databases. No plan caching would chew up your CPU. I don’t know.

Depends on if you’re, if you’re running high batch requests to second workloads, typically you have a bunch of pretty small queries running. It doesn’t take a lot to generate a query plan for a pretty small query. Great.

Like usually when you see like the, like the, like the, like the plans that take a lot of compile time CPU, they’ll, they’ll be like, there’ll be larger plans, right? They’re like plans a lot of exploration space, a lot of operators, lots of joins and whatever. So I don’t know. I don’t know about that.

If you’re doing OLTP with 4,000 lines for procedures, I think you’re messing something up. The single use ones truly eat up 32 procs. If you go to Eric darling data.com and click on consulting, we can talk more about that.

SQL dev DBA says smoke cigs in a French graveyard. Yes, please. That, that’s that, that, that, that is something that I would do eventually. I don’t know.

Uh, let’s see. Uh, do you have any insight on how the optimizer chooses per predicate for a single plan? Or is it time to nerd snipe? I don’t know what that means. Uh, how the optimizer chooses per predicate for a single plan. Well, I mean, per predicate, it depends on the cardinality estimator version.

It’s either like the, the assumption of a complete lack of cohesiveness, or it’s like the exponential back off thing in the new one. So it depends on, depends on the, the, the cardinality estimator.

Uh, let’s see here. Peter says, what do they say at the gym? Don’t skip recompile day. I’ve never heard that at the gym. Mostly. I just hear, uh, either, either vague words of encouragement or someone yesterday was at the gym. And, uh, uh, I was, I was, I had finished doing squats.

I was doing, I was doing overhead presses, getting, getting them up there, right? Getting them up. And, uh, and I was, I was in between sets and this guy, and this guy walked over to an empty rack and he was like, finally, this thing’s empty. I’ve been waiting like days for an empty rack. I’m like, that is a problem at my gym.

There’s a, there’s not that, there’s not that many racks and sometimes waiting for one sucks. But, uh, uh, then, then he, you know, I’m like, yeah, yeah, you know, it’s, it’s, it sucks waiting sometimes, you know, just gotta be patient sometimes, you know?

And like, you know, like all the, all the, all the racks are like platforms, right? So, you know, there’s like squat, like a squat, place a squat or do presses or whatever. And then like, there’s a platform in back where you could deadlift from. So, you know, it depends on what you want to do. Two people can use a platform at a time and like, you know, just kind of switch off.

Like I wouldn’t want to be deadlifting while some dude was squatting. That’d be a little awkward, but, but like, I’m feeling like, all right, well, this guy, this guy clearly needs a rack for something important. He’s going to get in there and do some, some rack stuff. And then he proceeded to, uh, stretch, do planks and then take, uh, a 25 pound plate and do like head circles with it, like back around this, doing this stuff.

And I thought, sir, there is not a single thing that you’ve done in the last 15 minutes. They’re required a rack. Nevermind you waiting two days to get on the rack. Go, you could do that at home. It’s not, it’s not fair.

It’s not fair. Oh, let’s see here. I got lost in that. Uh, Josh replied, Corey says, have you heard of event sourcing? Ever seen it implemented? Seems like a lot of temporal tables. No, I don’t think it’s a very popular thing. I don’t, I don’t think, uh, I would want to talk to anyone doing it.

It sounds, it sounds dreadful. Sorry. Uh, let’s see. Darren says squat day. I walked in and all the racks had weights on them, had to alter my workout, come to find they weren’t being used. Yeah. People do that.

That’s why whenever, whenever I see that, I just take a seat near the racks and I, I, I look at my watch for a little bit. And then if I don’t see anyone come use a rack, so much like a deadlock victim, I pick whichever rack has the least amount of weight on it. And then I take that off, figuring if whoever has the least amount of weight on a rack would probably be the easiest for me to like stare down.

If they were like, I was using that. It was like, no, you weren’t. You were off doing lat pull downs or something. Nerd. All right. Well, I’m glad that, uh, glad that Josh and Darren are friends now.

Maybe you guys can, I don’t know. What do people do when they make friends online? You like meet up for coffee or something? I don’t know.

I don’t know anymore. I don’t make friends anymore, especially not online. Chat rooms are for losers. Faris says, speaking of, speaking of chat rooms, Faris says, uh, if you had to pick a no SQL database to learn, which would it be?

Ah, boy, you know, I think just because of the amount of time I spend trying to convince people to use it, I would, like, like, I wouldn’t say like for life, but the first one I would learn would be elastic search. Cause I think the stuff that it does, uh, like as far, like as far as like how it does tech search versus SQL servers, crappy full text indexing catalog search is phenomenal.

I think there’s some great stuff that elastic search does. Also it’s free. Like, like free, free. You can go download it for free. You can install it for free. You can run it in production for free.

If you want support, it costs money like postgres, but like on the, like on the front of things, it’s totally free to just learn and deploy. And I think it would be fun to mess with. Unfortunately, I am a terrible developer and I would have no idea how to get data into it or how to query it whatsoever. None.

I would need the world’s dumbest API and like a pack of crayons in order to get anything done or working in there. It would not be, not be my best work. Peter says, uh, reading up a bit on Mongo.

Boo. Oh, come on, man. Mongo use, uh, what is it? Redis or react or what are the other cool ones? What’s that? It’s that thing.

Microsoft has in Azure, whatever it’s called. Cosmopolitan DB or something. Yeah. Josh likes Raven DB. Josh likes Raven DB because of one specific feature. Don’t let him con you into thinking that Raven DB is amazing.

The one, what Raven DB does is it tells you why certain indexes were skipped. And Josh thinks that that would be a healthy add on to SQL Server. I do too. I think that would be helpful. Like, especially during demos to be like, what in SQL Server use that index?

Here’s exactly why. Because it would impact the reliability. Microsoft sucks at error messages. I tweeted about some today. They’re, they’re God awful. I wish Microsoft should hire me to write error messages. Think it would bring some clarity to the situation.

They should hire me to write error messages. They should hire Joe. I wish to write KB articles. Uh, I think that, I think that, I think that would round things out. It’d probably go broke paying us though. We’d do a lot of writing.

At least I think anyway. Can’t believe this thing already has a thumbs up. Peter says, but then why would they need a consultant? Well, for what everyone else needs consultants for, Peter, to supplement exist, exist, existing staff that has some very clear weaknesses.

That’s what. Whoever writes those error messages now, oh boy, you need help.

Almost a complete lack of clarity. Punctuation, capitalization. It’s wiggity whack, yo. Real wiggity whack. All right.

We are coming up on the half hour mark. And I’m seeing a distinct lack of sentences that end in question marks. Or comments that end in something else. So, Josh says, Eric would replace all of the sys messages with sounds like a string problem.

Perhaps, but, I would at the very least point people in the right direction. It wouldn’t be that, that please retry. Nonsense.

I would give people, I would say, yeah, I think you messed this up. Here’s what you should do instead. Like if someone tried to, I don’t know, duo, do, do, do things that are bad before SQL Server 2019. I would throw in error messages. Like if they tried to create a scalar function, I would, the first, then the first create, I would, I would throw an error message.

They’d be like, you sure? Look, this is what’s going to happen. It’s going to be bad if you do this. Unless you’re a consultant just writing a demo about them. You should avoid that. Yeah. I am too old for this. It’s time to go smoke cigarettes in a French graveyard. So, thank you for joining me this week.

I will, at this, at this, at this rate, I think I will be, I will not have anything in the way, next Friday. So, I should be able to do that. Actually, I would change all the error messages to don’t at me. That would be, that would be it.

Anyway, thanks for coming along. Thanks for watching. See you next time.

Going Further


If this is the kind of SQL Server stuff you love learning about, you’ll love my training. Blog readers get 25% off the Everything Bundle — over 100 hours of performance tuning content. Need hands-on help? I offer consulting engagements from targeted investigations to ongoing retainers. Want a quick sanity check before committing to a full engagement? Schedule a call — no commitment required.

Last Week’s Almost Definitely Not Office Hours: April 19

ICYMI


Last week’s thrilling, stunning, flawless episode of whatever-you-wanna-call-it.

Was preempted by a flight! Sorry about that, we’ll be back next week.

The perils of being a one man band, I suppose.

Going Further


If this is the kind of SQL Server stuff you love learning about, you’ll love my training. Blog readers get 25% off the Everything Bundle — over 100 hours of performance tuning content. Need hands-on help? I offer consulting engagements from targeted investigations to ongoing retainers. Want a quick sanity check before committing to a full engagement? Schedule a call — no commitment required.

Last Week’s Almost Definitely Not Office Hours: April 12

ICYMI


Last week’s thrilling, stunning, flawless episode of whatever-you-wanna-call-it.

Video Summary

In this video, I found myself reflecting on the challenges and frustrations of working in IT, particularly when it comes to database management and moving between different systems. The conversation touched on a range of topics, from the excitement surrounding SQL Server 2019 to the difficulties of convincing users to upgrade from older versions. One of the key points was the disparity between the features available in newer versions of SQL Server and the reluctance of many organizations to move beyond their current setups. This led to discussions about memory management, trace flags, and even the transition to alternative databases like PostgreSQL on Linux. Despite these challenges, I shared my experience with a specific instance where setting up lock pages in memory on a laptop helped silence error logs but also highlighted the complexities involved when managing resources across different environments. Overall, it was an engaging session that underscored both the benefits and pitfalls of working with complex IT systems.

Full Transcript

I’m live. I bet no one’s going to show up this week because I missed two weeks in a row, which is terrible of me. Terrible of me. What a horrible person I am. Let’s see here.

Uh, yeah. Still no one. Not to. I’m going to wait one minute and then I’m out of here and I will delete this from memory. Oh, there’s one person.

All right. All one of you. I hope you have lots of good questions. Otherwise, otherwise we’re going to be very happy to be able to do this. I’m going to be very lonely. Oh, God, it’s you. Oh, man.

It’s going to be like insider chat when I just talk about funny stuff that happened in Madison. All right. Two of you.

Two of you. Not bad. All right. Not bad. Not bad. All right. Man, this is what I get from missing time. Messes and messes everyone up. That and my camera refuses to focus. Why are you so close?

I’m back here. Oh, God damn it. There we go. All right. Camera’s sort of working now. Maybe. Back to one. Crap. Someone ask a question.

Someone say something. I’m going to be like this is getting out of hand. Getting way out of hand. Ba-bum.

Ba-bum. Ba-bum. Ba-bum. Ba-bum. Ba-bum. Ba-bum. Ba-bum. Ba-bum. Oh, man. All right.

This is dull. Forrest is here and not asking questions. No one else is showing up. Actually, I can’t even see who’s still here. That’s the depressing thing. I don’t have like an attendee list. All right. This gets one more minute.

Not even a minute. This gets 30 seconds. And then it’s dead to me. Dead to me. Uh… Fifteen. Ten.

Uh… Two people. Is one of you going to do something interesting? I don’t know. I don’t know what’s going to happen. What’s going to happen? I’m not a squirrel to sit here.

I’m tuning the worst door procedure in the world right now anyway. It is like eight CTE chained and joined and bound and gagged. And the query plan is like this big about. Is that a hole in my shirt?

Oh, it is. Man, I look terrible now. I didn’t know that was there. Anything over here? No. All right. We’re getting rid of this shirt when we’re done. Now you can all see the one spot on my body that I don’t have a tattoo. Sad. We have this like just running this thing to like get a plan for it and compare other things to it is absurd.

It takes, I don’t know how long. It just runs for a very long time. Doing a lot of dumb things. Doing a lot of things that I don’t necessarily agree with. I was in Madison last week.

And I had a client thing the week before. And thankfully I have a client thing. After this. At one. So I’ll be bothering.

Bothering people for money instead of bothering you for free. It’s always a good call. And then. Yeah. So Madison was a lot of fun. I had like 50 people. When my, my pre-con.

It was sold out. Wild. A lot of people in there. And the event. The event itself was pretty fun. I got to see.

Joe Obish talk about columnstore. Which is a treat. Got to see. I don’t know. I forget. I forget what else happened. Yeah. Selling stuff out is a, it’s a fun, fun feeling. But then it makes you think like, man, what if there were, what if there were 10 more seats?

You don’t get greedy, but you’re like, oh, what if? What if? What if, what if that room was a little bigger? What could, how many more people could we have taught about SQL Server and taco consumption? And the, the flight back.

Well, I, before I, before I talk about anything else, if, if you’re able to attend SQL Saturday Madison next year, I highly recommend it. Especially if they invite me back. Yeah. The flight back was, was weird.

Because what happened is. They, they, they, they thought that there would be unexpected, they expected unexpected turbulence. And they stopped drink service.

Yeah. So that was, that was a bummer of a flight. Even, even sitting in the, in the upfront section, I couldn’t get, I couldn’t get another bottle of, another little mini bottle of wine. And I was very sad about that. And so I was, I was sitting there just waiting.

That was, that was fun. Farah says, if you were a client, how would you pick a quality consultant? I would go to ericdarlingdata.com. And I would click on the, the contact form. And I would, I would send an email to a blurry, a blurry gentleman whose camera refuses to focus.

Well, I mean, really, it depends on what you, what you want out of consulting. There are some people who want, you know, sort of long-term remote DBA work. And, you know, someone to provide on-call support.

And, and for them, I think Mike Walsh over at Straight Past SQL is a pretty good choice for that. There are some people who want advice about their SQL Server. And that’s where someone like me comes in. I’m very good at giving people advice. I’m not good at giving myself advice.

Clearly ended up with, with this, with neck tattoos and this haircut. So I don’t really know that I would give my, I would take advice from me about me. But advice about SQL Server for someone else, I’m, I’m pretty good at. But then there’s people who want like, you know, sort of someone to jump in and start tuning things. I can do that.

And I’m, I’m, I’m perfectly comfortable doing that. You know, if you have a performance tuning stuff, I’m happy to do that work too. But if you need to like advice about H A and D R, I would probably choose someone other than me. Because I’m going to tell you the same thing I tell everyone else.

Get a failover cluster and get log shipping and don’t think about anything else. Zane says that he would take advice from me about lifting large objects. The funny thing is, is I lift fairly small objects that just happen to weigh a lot. So if I had to lift something big, I don’t know.

I don’t know how I do. I don’t know how I’d handle that. Peter says, why even fly if there’s no booze? I feel the same way. Man, when I, when I, when there’s a, when there’s a plane trip involved, it’s me. Like I get to the airport like two hours early.

I’m hanging out in the lounge. I get on the plane and I’m like, like slam the car down. Like give me all the red wine you have. I’ll give you back whatever I don’t finish. And, and that’s pretty fun. Let’s see.

I don’t know. Let’s see. Peter says, how, how is it settling into being, being your own CEO? Pretty good. Pretty good on that. Look, I got, I got, I got a thing. Yeah. Look, it’s got my name on it. So I’m pretty happy with that. The card number isn’t on the front. So don’t bother trying to pause that to figure anything out.

The card number is on the back. I checked. So you can’t tell anything from that, but I’m, I’m, I’m doing, doing pretty good as my own boss. At least I’m, at least I’m pretty happy with my performance as my own boss. I don’t know how other people feel about it. We’ll have to wait and see.

I got, I got my first wireless phone charger. Cause the battery port on my like thousand year old phone is busted. They’re not busted. It’s just very flaky. So it’s a lot of like me just like wiggling wires to get things working. Anyway, should probably answer a SQL Server question at some point.

If anyone has one, I don’t know if anyone has one. What do I think about SQL Server 2019? Yes.

I know who you are. You’ve been, uh, from what I’ve seen of it, I like it. I know. I mean, it’s only in, it’s only in CTP and it’s only CTP 2.4. And you know, it’s still good. It’s still good. It’s not even like a, it hasn’t even hit release candidate yet. And I have a feeling that it’ll be at least one more, if not two more CTPs before we hit release candidate.

But so far it’s very promising. You know, um, it’s, it’s got some nice steps forward in a lot of ways for things. And it’s going to be interesting to see in, in six years, how those steps forward finally pan out when people start moving to 2019. Because damn you people can’t even get off 2008 and 2008 R2.

How am I supposed to get excited about 2019? How am I supposed to get excited about it? What demos? Man, it’s rough. I’m like, look at all this cool stuff you have. Look at all this cool stuff you have in 2019.

People are like, eh, I got 2008. I’m like, man, it’s brutal. It’s brutal out there. Like, I don’t know how, I don’t know how to talk people into doing something smarter with their, with their lives and with themselves.

It’s very frustrating. Very frustrating. But no one listens. No one listens to consultants anyway. It’s like, what do you want me to do? What do you want me to do? Let’s see.

Peter says, I’m moving from SQL Server to Postgres on Linux starting Monday. Please tell me why I’m insane. I mean, it’s, it’s crazy because, you know, uh, you know, I actually, you know, I don’t know. You, you, you, you might be really into Postgres and you might be really into Linux. I, um, I once took a job very briefly administering.

Well, the, the job description was like, it’s Postgres SQL. And I was like, cool. Well, I I’ve heard of that. It has some neat stuff in it. It’s developer friendly. I’ll work with that.

And then I, then I got in there and it was red shift. I was like, oh, that’s a little different. It’s like a, like kind of a fork of an older Postgres, but I was like, I’ll, I’ll stick with it. Let’s see what happens. And then it was just like, well, we’ve got these consultants. And I was like, oh boy. And the consultants are like, we’ve built an ETL process.

And I was like, sweet. I don’t have to do any work. And then I was like, is there any documentation for the ETL process? And they were like, no. And I was like, could you make some? And they were like, we’re not full-time employees. I was like, what? And I was like, well, can you, can you please just like, give me like a brief overview of how to use the ETL tool that you built?

Because it was, it was not like a command line tool where you could do anything and things were exposed to you. It was a command line. I mean, sorry, it wasn’t a GUI. It was a command line tool. And you had to like do a bunch of stuff to the files to get it to work. And none of it was like, like written out.

And so I’m sitting there and I’m like, all right, look, you need to document. This in some way that I can at least like get off the ground with it. And they were like, okay, fine. And they made a video. It was a 20 minute video.

And there was no voiceover. There was no voice explaining what was going on. There was just a mouse pointer circling things and like pointing agitatedly at things. And the audio was like a country, like, like diddly guitar loop was like, or something like that.

But it was like that over and over again. And I’m sitting there listening to this and watching this like frantic mouse over. And like these, like the video was on like, like this big, and I couldn’t even see what was being written into the command line. And I quit soon after that. I got a new job after that. It was like immediate.

It was like, I was like, like, as soon as like I’m watching this video on one screen and like working on my resume on the other screen. And I think I lasted like, I think it was like a month and a half, maybe two months there. And it was just like, this is like, and I would have been happy. I’m not happy.

I would have been okay working with like Redshift and doing stuff there. Cause it’s a pretty cool tool, but man, it like the, everything else that was going on around, around that, that project was. I don’t know a lot of German curse words.

So I don’t know. Let’s see. Laura says thoughts about memory management. In other words, enterprise edition and eight gigs more of memory plus lockpages of memory. Wait, eight gigs of memory. Why do you have eight gigs of memory on enterprise edition? That’s like, like my, I think my phone has most of eight gigs.

Why? That’s not, not, not rational. It’s not sane. Don’t do that. Yeah. That was, that was the hamster dance song. Why do you, okay.

I don’t understand what your question is. And what you have a lot more memory than my eight gig minimum. I don’t care. That’s fine. Does it, does it never bothered me any minimum memory, setting minimum server memory to eight gigs.

I guess. If you have lockpages and memory on, you shouldn’t need to set minimum memory. Uh, and, uh, yeah, I, I, I’m, I’m trying to figure out a way when like lockpages and memory would be a worse choice than setting min server memory. And the only time that it’s like, either the only time, so be fair, either of those settings will make things go kaboom.

If you have an overcommitted VM and that memory balloon driver starts freaking out. So be careful with that. If it’s a VM and the VM host is severely overcommitted, then, uh, I would, I would avoid, I would avoid either of those settings, but setting min server memory to eight gigs. I don’t think is gonna really do much of anything.

Do you use trace like eight 34? Uh, no, not really. Um, no, uh, I’ve never really found, I’ve never really found a, a, a, a use case for, for me anyway, where it made things significantly better. Um, so I’ve never, I’ve never like really set it up and kept it.

It was, it was like, like sometimes I tried stuff and I couldn’t, I couldn’t find a tangible difference. There might’ve been some like cool spin locky under the, under the covers difference that, you know, um, uh, I don’t know. Maybe, maybe someone else found something cool to do, but, uh, I could never figure out what, what would be fun or interesting to do with that.

So, sorry, I wish, I wish I had better advice about trace like eight 34. Uh, I know that there was a bunch of, um, bunch of guidance about when to use it. And for what, like a while ago, I just don’t know.

I just don’t know that it’s terribly relevant anymore. See, Peter says, I set lock pages of memory on my laptop just to shut up the error log. Uh, I set lock pages in memory too, but, uh, then sometimes I have to do stuff that requires memory for other things. And I have to like restart SQL.

So it gives up the memory. So like, oh, my, like on my laptop, I have 64 gigs of memory. Uh, my local server instance can get like, I think 50 gigs or something or 55 gigs. I forget what I haven’t said to. But then like, if I have a, I have a VM that I set to get like 16 gigs or eight gigs or something, eventually I can’t start my VMs or like the VMs start freaking out because SQL Server is just like, nope, that’s my memory. You can’t have any, you don’t get any of that bad DBA.

Okay. Let’s see. Uh, all right. Nothing fun going on there. Uh, so I’ve, I’ve started taking, well, I’ve started offering to take questions for office hours via like Twitter and email and stuff. And, uh, so far I haven’t really gotten any.

The one, the one question that I got was like, like, like, like, like, like a page long thing. It was like, you should tell me how to like do this very specific thing. I was like, oh yeah, that’s consulting. You should, should pay. Can’t answer that in a half hour of free webcast.

Let’s see. Uh, he just says about half the time I have SQL Server as manual start. No, I have it as manual start too, but then I start up and I do stuff with it. And then I have to switch to use a VM to do something. And I hit VM problems. Is there a hashtag or something?

No, just tweet me, whatever it is, whatever you want to call it. I don’t know. Whatever, whatever you call the form of communication of, on Twitter, where there’s an at sign and then my Twitter handle, it would be a fine way to get to things. I think.

I think anyway, I can’t, I can’t say for sure for everyone that that would be a good thing. And some of you might not even be on Twitter. I don’t know. Some of you might, might do other things that, um, that like, I don’t know, make you happy in life. Or I don’t know.

What else? What else? What do people do who aren’t on Twitter? Like what, what’s your internet? Like, what, like, what do you, what do you do with your internet existence? I’m curious. Like, it’s stuff that like, I would be interested in. It was like, cause like I need, cause now I need stuff to do that’s not Twitter.

And that would be, I think appropriate for me to have like, just non-Twitter things going on. So if you have, if you have non-Twitter things, that would be great. Like, like going outside.

So, yeah. So, but see, the thing is, every time I go outside, it’s expensive. When I go outside, it’s like, uh, I, I go eat food. And when I go outside and eat food, that’s expensive.

Cause I have a wife and two kids and I got to drag them everywhere. Or if I, even if I go, if I go up by myself, I’m like, well, I’m going to go someplace real nice. Now they don’t have, I don’t have those losers. But no, I was kidding. Uh, but yeah, going outside. And then like, I go to the gym that’s going outside, but the gym isn’t far away. I, I, I’ve done an admirable job of, of going to the gym in the laziest way possible, where the gym is like a three minute walk.

So Laura says I uninstalled Twitter on the phone to let go of it. Sometimes that’s a good, that’s a good call. Um, for the, for the first like month or so, I was just like, you know, I’m only going to, uh, have, I’m only going to use Twitter on my phone, like through the website.

And then I realized that like, sometimes that was just a really, really bad way to be doing things. Like I spent longer wrestling something on the website than is this really quickly, quick to do with the app. And so I don’t know. That was it.

It says what technical awesomeness did attendees here? Madison precon most appreciate. I don’t know. I haven’t gotten session feedback yet. Um, I think some of the stuff about how SQL Server allocates threads and memory to queries went over pretty well.

And I think, uh, people’s people get a really eyeopening, um, experience from, uh, when I show them exactly how contention works with, uh, sans. So like showing them like that, the wire is a limiting factor and that like, like contention there, like the, uh, like the, the competition for a shared resource is what often makes sans feel slow when it’s not, but I shouldn’t, I can’t give away too much.

Then, then no one will ever pay to see it again. I’m surprised they paid to see it in the first place, but if I get, if I get to have that happen again, that’s wonderful. So there’s that, I don’t know. There was at least one attendee of my precon in this room. So maybe they can, maybe they can tell you more about what they, what they liked about it.

Yeah. Dang wires. I hate those dang wires. Uh, is the three minute walk to the gym, my car. Yeah, basically. Sometimes I, uh, yeah, that’s about it. Yeah.

Yeah. I don’t know. Occasionally I wrestle with my humanity. That’s a, that’s a, that’s a, that’s a quick fight. Humanity never wins. Uh, let’s see here. Darren said, uh, wait, wait, there’s another, there’s actual, see what’s it?

Uh, actual last query plan. Thoughts. Um, no, I, mixed feelings. Uh, one, one of those mixed feelings is like. All right. If you’re going to invest in that, put it in query store, because that’s the kind of thing that would make people turn query store on.

But at the same time, if it’s in query store, most people are going to go, I’m not turning that thing on. Um, but I think it would, it would talk some people into it. I think it would be, uh, work. I think that would be a worthy thing there. Um, it doesn’t really collect actual, actual stuff that I would want inquiry plans like operator times and like weight stats and stuff like that.

So then that, that, that, that would be really compelling for me to start using it. Uh, you know, it, like, um, like as soon as I saw it, I was like, man, it would be like awesome. If I could like, you know, tweak blitz cash to look at that DMV instead and get all this actual query plan information out.

And then I saw what was in it and I was like, eh, that’s not actual enough. Not actual enough for me to get into GitHub again. So yeah, that was that. Uh, let’s see here. Any SQL Saturday pre-cons, any other ones this year? Um, no SQL Saturday pre-cons lined up yet, though I am open if anyone out there is listening and, and is in need of, uh, uh, uh, a funny performance pre-con, uh, at their SQL Saturday event, hit me up.

There’s many ways to do that. Um, let’s see. Uh, I don’t know how to say that. Uh, do you listen to music or podcasts while gymming? No, I, uh, headphones just get in the way. Uh, if you, if you get, if you’re, if you get bored at the gym, you’re doing cardio and I have no sympathy for you.

If you’re lifting weights, you don’t get bored. If you’re lifting weights here, you’re too, too encumbered to get bored. You gotta, you gotta really do something weird with weights to get bored lifting weights. He says, what percent is of your clients don’t use query store so you can’t convince them to turn it on?

Uh, I don’t know. Um, I can convince most people to turn it on, on a dev server and collect information on things. Uh, but, you know, in prod people are a little bit more prickly about what they turn on and what they use.

Josh says, I wonder if more people would turn on query store if the default capture mode wasn’t all. Uh, so maybe, well, no, no, cause when most people turn it on, they have no idea that there’s a default capture mode. Problem is it like no one ever, it’s like, it’s like nothing about when you go look around at stuff, you’re just like, oh yeah, turn on query store.

There are options. I’m not gonna, I’m scared of those. Um, but even if it was auto, I mean, like people turn it on and they have all and it’s like collects everything and maybe burns a bunch of CPU and everything stinks.

And then they turn it off and they’re like, well, that sucked. Never turning that thing on again. Tech news says, where are you from? I’m from America. Where are you from? Tech news. Tech news.

Anyway, like query, and query store is one of those things that I’m afraid is just gonna like hit a wall. Cause this query store DMVs are terrible. Like the information they collect is great, but querying them is a nightmare. The built-in reports are janky as hell.

They take forever to run. And I don’t necessarily blame like whoever wrote the reports, but you know, you’re, you’re, you’re hitting these like, you know, I don’t know what you want to call them. Um, base views or tables or table type functions or whatever. And it’s just like, Oh, it’s a very, very slow sometimes.

And you’re churning through a lot of data and you’re, you know, you’ve got query plans coming up. You’ve got query texts coming up. And I just feel like these are things that are not as difficult to, to deal with in the plan cache.

But for some reason, query store made them even harder to deal with. Right. Tech news is from India. I’ve never been to India. I might go there someday. I should, I should someday go to the homeland of my favorite food on the planet. Which David Lynch Dune version?

Are you, do you mean like, like between like the regular version and like the extended cut and the director’s cut or like the TV show? Like which, which versions, which versions are on, which versions are in play here? Yeah.

Yeah. I like it. I like, I like it for as long as I can get it. And I’ll tell you why. Most of the time these days when I get to watch Dune is when my wife and kids aren’t home. And I usually end up taking a nap while I watch it. And I feel like if I fall asleep at the beginning of Dune and I wake up at the end of Dune and it’s like the director’s cut, I’ve taken a good nap.

That’s like a solid, like, like two hours. Of nap time. So I prefer the director’s cut for that. If I, if I wake up, if I use the regular version and I wake up and Dune is over, I have no way to gauge how long I slept. But if I wait, like fall asleep, like when they’re, when they’re doing like the intro stuff.

And then I, I, I wake up when like the, like when Paul is fighting, whatever Sting’s name was, I forget now, whatever Harkonnen that was. Then I’ve had a good nap. Uh, TZH says being able, unable to do things like click on a query in the query store UI and saying, I really don’t care about this really limits its useless.

I’ve tried to find the DMVs instead of the behind it, but as far as I know, they’re really not friendly. I mean, they’re all, they’re all available to you. Like you can find them on, on books online, but yeah, they’re, they’re really unfriendly to query. Um, and the, the, the way the data is stored is kind of annoying to deal with.

Cause if you want to put like any of the metrics in human consumable form, you have to like do some math to convert stuff from like microseconds or from 8k pages. And you’re like, man, just tell me what’s bad. You have all my queries.

Tell me what sucks. You have all my query XML. Like, like tell me what’s bad in it. Why, why do you make me go jump through all these damn hoops to do things? And I feel the same way about monitoring tools that, you know, uh, grab like endless server information and then like give you no guidance about what to do with it.

They’re sitting there watching your server. They’re taking information constantly from your server. They’re collecting metric after metric. They’re offering you absolutely no solutions. Just sitting there watching they like, and they have, they can like, they can asynchronously same thing with query.

So can asynchronously just look through like a little bit of XML at a time. Look for like some bad stuff in there. Look for non-parallel plan reasons. Look for implicit conversions. Look for missing indexes. Look for, for anything. Go in there and show me what’s wrong. Tell me what’s happening.

And it drives me nuts. It’s software that makes your life not any easier. Software should never, never like leave you like where you are. Use software to make your life better, to make your life easier. Using software that makes your life harder or doesn’t actually solve a problem for you. Especially when you have to pay for it.

Sucks. Sucks. Fix your software. Sons of guns. Maybe, maybe there are some daughters of guns too. I don’t know. I have a whole family of gun relatives. Fix your stuff.

Get your act together. People aren’t going to pay 1200 bucks per instance for garbage for long. Tell you that much. Let’s see. Peter says, I was told by a guy with neck tattoos and implicit conversion warnings might be a red herring. Yeah, that’s true. Some of them are. And there’s very easy ways to tell the difference between which ones are and which ones aren’t.

So seek affecting, which is in the XML. You can see when it’s seek affecting. Then, perhaps you have an issue. But if it’s just the cardinality estimation one, that happens when you select stuff and it converts and you’re like, wow, how is selecting something is a different data type going to mess up cardinality estimation?

You know damn well how many of that is coming out no matter what. It’s like the no join predicate warning. It’s absurd. There’s no join predicate, but there is.

And it’s just because there’s not one at the join that it freaks out. It’s like, okay. You really got me there. It’s solving a big problem for me. Oh, man.

All right. I got a client thing starting in a half hour, so I got to go. Thank you all for showing up and asking me questions. I will be back next week. I hope. You know, assuming good health and lack of death continue to shine upon me. I will be back next week.

And yeah, I promise I’ll actually even do some promotion for it. Who knows? It’ll be crazy. Anyway, thanks. I’ll see you next time. Goodbye. Goodbye. Thank you.

Going Further


If this is the kind of SQL Server stuff you love learning about, you’ll love my training. Blog readers get 25% off the Everything Bundle — over 100 hours of performance tuning content. Need hands-on help? I offer consulting engagements from targeted investigations to ongoing retainers. Want a quick sanity check before committing to a full engagement? Schedule a call — no commitment required.

Last Week’s Almost Definitely Not Office Hours: April 5

ICYMI


Last week’s thrilling, stunning, flawless episode of whatever-you-wanna-call-it.

Thanks for watching!

Video Summary

In this video, I share my experiences from the recent Bits conference and reflect on the fun and challenges of being a speaker there. I discuss how I managed to keep my mustache despite my wife’s initial approval, hinting at potential adventures or hijinks that might come with it. Additionally, I delve into some technical topics like foreign keys in production, deadlock issues related to cascading actions, and troubleshooting SPN stuff for SQL Server. I also provide insights on SSIS package design and the behavior of spools in query plans, hoping to help fellow database professionals navigate these complex areas more effectively.

Full Transcript

I’m alive. I’m live. Oh, I’m so nervous. It’s been like 10 seconds and no one’s here. Let me quit things that make noise. So I’m back from bits, and bits was quite enjoyable. I like bits a lot. I hope that I get to go back next year. Regardless of where it is, who knows where it will be? I know that right now they kind of rotate between a few different places. So it’s like Telford, London, Manchester. Last year was London, this year was Manchester, so I don’t know if it’ll be back in Telford. Though I hear there’s not much ado. in Telford, England. So who knows? Maybe they’ll find somewhere else to do it where there’s more going on. I ran out of stickers while I was there. I had to get new stickers. I ordered much more this time. Many more stickers. I actually have to send some out to people. And I have fun secret swag coming that will hopefully be here in time for a… I got a pre-con in Madness.

in. Yeah. I went to Madsson, Wisconsin at the beginning of the month. Next month. And like the sixth I think. So I’m going to be off to Madsson, Wisconsin to talk more about SQL Server. I’m gonna see my friend Joe Obish who lives out there. Be a good time for everyone. I hope. I still haven’t been able to shave off the mustache. My wife decided that she likes it. Yeah. Yeah. Keeping it on for now. Wife might, I was gonna shave it off. I was like, yeah, keep it look good. I was like, I don’t think so, but you said so. You know, what the hell? What’s the worst thing that happens? I have a mustache. Where I live, it’s not the weirdest thing in the world for someone to have hand and neck tattoos and a mustache. Stick with it. See how things go. Who knows? Maybe this mustache will get me into some trouble, get some hijinks, have some adventure in my life. The mustache adventures.

Food crumbs. More like leftovers, yes. You’re talking to a guy who has had food on his glasses on several occasions, so food and a mustache wouldn’t be too outlandish. Wild stuff out there. All right. What was I going to say? I’ve been working on some fun stuff this week.

I found some fun deadlock things, and the deadlocks were all related to cascading foreign keys, and many of the cascading actions, cascading deletes more specifically, many of the cascading actions didn’t have supporting indexes for the foreign keys, so there’s like these giant clustered index scans hidden away in foreign key cascading actions, which is quite interesting. And I’ve been writing some weird queries against the Sentry 1 repository database, so like some of the stuff that’s not available in the GUI yet is like hidden away in the table, so I’ve been writing stuff against that. I want to blog about it, but I’m not sure where the blog post is going to end up. I don’t know.

It depends on what Aaron Bertrand thinks about my query. Let’s see where that goes. Fun stuff there. Getting emails from people about stuff and things. Someone ask a question, because I’m just sitting here babbling to myself for the last five minutes. If no one has questions, I’m just going to go put my head down, because I have a terrible sinus infection in case you can’t tell. My face feels awful. Worse than it looks, I guarantee you.

Any recommendations for automated ERDs other than database diagrams and SSMS? Well, SSMS is going, got rid of those. They’ve been deprecated. As soon as you move to SSMS, I think it’s version 18. They’re gone. I think Toad makes a pretty good one.

Richie used to have a lot of good recommendations for these. If you’re on Twitter and you want to bother Richie about it, then I would say, hey, Richie, what’s good for database diagrams? He used to use those a lot for stuff. Me, I just model everything after the Stack Overflow database.

Yeah, and that’s why they’re getting rid of them, I guess, because hardly anyone uses them. And everyone I know at some point in their life has opened up Management Studio, went to click on something and accidentally clicked on database diagrams. You get that pop up, like, there aren’t any diagrams. Do you want to make something? You’re like, no, I didn’t want that at all.

I wouldn’t like the opposite of that. I would like no diagrams. So I had something kind of funny happen the other day where I went to the archery range and I was shooting for the first time in, like, months. And I did not move my arm out of the way in time.

And you can’t really tell all that well because of the tattoo and everything, but there’s this big bruise going down my arm and it’s swollen as hell because a bowstring caught it. A lot of fun there. A lot of fun there. Let’s see. Josh asks, how would you explain CX consumer weights to someone that is so new? Well, CX consumer is like the leader of a gang, right? So you have, let’s say that you have a dot for query running and you have four worker threads and they go out there and do gang stuff, go out there, do some crime, right? Doing awful things out there to good law abiding citizens. So you have people, you have these four gang members out there doing awful things. And that one coordinator thread is the gang leader and he’s waiting for them to come back and hear their stories. That’s CX consumer. CX consumer is a gang leader and he’s waiting on all the other gang members to finish doing things. They come back from their nefarious deeds and activities. So he sends them out and they go out and do things. And then, well, those four, four people are out there doing stuff. CX consumer is racking up saying, come on, man, hurry up, hurry up, bring my money. Smacking people around.

Zane says he used it once and it didn’t work so well. I agree that that has never worked so well. So let’s all agree not to use that ever again because that, that doesn’t sound like a lot of fun. Uh, let’s see. Uh, I made foreign keys in production unknowingly with the diagrams once.

Wow. That’s, uh, that’s interesting. Did, cause I’m assuming that that didn’t like, like it just created foreign keys and it didn’t actually like, like index them or do anything helpful to support those foreign keys. Yeah, no, right. No, probably not. Yeah. That’s a good time. That’s a real good time. Uh, yes, you’re welcome, Josh. Um, I, I have no other insight into that. Um, I, I did blog a while back. Some guy, you’ve probably never heard of a site, um, about, uh, about when, cause a lot of people said that CX consumer weights were not, uh, or something that you can ignore. And I disagree completely because, uh, if you have a parallel query that is waiting on a lot of CX consumer, it can be, you’ll have to excuse the crime scene back there. Uh, it can be, um, what do you call it? Uh, bad, but it can be a sign of, uh, really, really terribly skewed parallelism. Uh, Julie asks, what’s the best way to troubleshoot SSPI SPN errors? Uh, I skip right past troubleshooting and go right to the active directory people. And when I talk to the active, active directory people, I say, Hey, active directly people, make sure that my SQL Server service accounts, uh, have the ability to delegate SPNs. I want them to specifically have that privilege. Uh, so that I don’t have to troubleshoot that error. I just know that my, my AD service accounts have that privilege available to them. And I don’t have to think about it. There are like all sorts of, you know, uh, commands you can write to set like, uh, DOS commands and PowerShell commands to like set SPNs manually. Uh, that’s for the birds. I say active directory person, please give my, uh, account the ability to delegate SPNs on its own. So I don’t have to worry about that later because I’ll be damned if I’m gonna remember all those commands. Um, if anyone wants to be super helpful, uh, the late great Robert Davis had a blog post about, uh, troubleshooting SPN stuff. And if you feel like Googling that and putting the link in chat for me, that’d be great. Cause when I start doing that, I get all messed up. I don’t have anyone to help me with it. I need, I need, I need all you to be helpers.

Hugo says at least you had foreign keys better than most databases. I don’t know. I, I almost disagree because most of the time I’m not getting any great benefit from foreign keys. Like it’s nice to know the relationships, but I’m not seeing like, uh, what do you call it there? I like join elimination. Cause how often are you like querying two tables and only selecting columns from one? And then, you know, you load data in or like you have to delete data or like you have to update data or anything like that and go start checking all those foreign keys for the birds and the birds. Julian was Robert Davis, D A V I S. Uh, you’ll, his website is SQL soldier dot something calm probably. But over there, there was a, a good, a good writeup on a SPN type stuff. But like I said, you’re much better off going to the, uh, the AD people and just requesting the right permissions for your, your account, your service accounts. Let’s see. Hugo says more often than you’d think through views. I views. What’s up next? You have a lot of tricks up your sleeves. Hugo’s got tricks.

I hung out with Hugo at SQL bits. Hugo’s always fun to hang out with. Hugo is sitting in my session right behind it. So it’s just like no pressure there. I am dressed up in a, in a wife beater and, and, and, and, and, and light wash jeans running around like a, like a, like a fool and staring at these two. And they’re like the front row, like terrified. Like, please don’t let me say anything. Don’t let me say anything. Don’t. Uh, Zane says most of my work has been OLAP. And then FKs have not seemed necessary or beneficial. Yeah. Uh, data warehouse foreign keys. I’m, I’m, I’m usually, I’m usually pretty steadfastly against those. You might be able to convince me in a few, few scenarios, but, uh, for the most part, you know, uh, if I see foreign keys in a data warehouse, I get nervous because if you, especially if it’s, if, especially if it’s like a clear out and reload data warehouse, where you’re like, not just like a trickle in, it’s got a slowly trickling, changing thing. If you have to, if you’re just like every day, you’re like wiping it out and reloading lots and lots of data in foreign keys, but your butt hard, not, not good there. If you like data integrity should be done in the OLTP side, you shouldn’t be doing data integrity in the data warehouse. That should be handled when you put data in, like in tiny little chunks.

Let’s see. Uh, what is that? Uh, uh, let’s see. Uh, that’s a long name there. Uh, I am working on an SSIS package that identifies non-matching records in a, in a progress DB, uh, and updates records in Azure SQL DB. Would you create a new SQL DB for the progress table and then use a lookup task or is there a more elegant solution? Uh, I’m going to be very honest with you. I do not use SSIS a whole lot. Um, if you have, if you, if this is a new project and you have the luxury, I would, I would try it every way that you think might be, might be rational for you to go with and, uh, see which one works the best. Um, you know, lookup, lookup tables are certainly good for some, man, there is someone out there sawing away. I hope it’s not like a mass murder. I hope it’s not uh, I don’t have a terribly good answer for that just because of my, my lack of experience with SSIS.

Uh, I’m not really sure what the workflow you have currently looks like and why you think the, the lookup table would be better. If you want to stick some more details, maybe some nice person in chat who uses SSIS a lot would chime in and save my skin from, from, uh, more babbling about things that I don’t know. Oh my goodness. Let’s see. I had an email question or rather a, a, a Twitter message question this week about spools. And, uh, I’m waiting for the person to send me the query plan for it. But the question was, why do spools spools so much data? So like, that was, that was like the, the, the bottom line of the question. And the reason is that, uh, a spool doesn’t just, uh, so a lazy spool doesn’t just execute once. An eager spool will execute once, grab a whole bunch of rows, and then allow the parent operator to just kind of grab whatever it wants from that spool. Uh, lazy spools execute lots of times. Usually you can tell by the rebinds and rewinds and executions. Well, actually executions will be the total of rebinds and rewinds for a lazy spool. But, uh, for a rewind, that means you, uh, you use data in the spool.

And for a rebind, that means you went down to the child operators of the spool, ran those and got a new set of data and brought that in to the spool. And the spools usually happen on the inner side of nested loops because it’s a very repetitive, right? It’s a loop. It’s the only, you only join that loops, hashes and merge joins. They go get data, go get data, jam that data together. Nested loops are like, I got some data, go look, get some data, go look, get some data, go look. So it gets very repetitive.

And when the optimizer says, I think we could make this repetitive task less repetitive by reusing spool data, then it creates a spool of some kind, either, usually either a table spool or an index spool. And it, uh, and it populates it with stuff, usually data. And then it goes and uses that data. So for a table spool specifically, a lazy table spool, uh, you’ll, you’ll get a value from the other side of nested loops, say, go get me, go spool this data for me. Usually there’s a sort in there at some point too, if your data doesn’t, if your data isn’t in, isn’t in index order, like, uh, of the way you’re going to go look for it, uh, the optimizer will usually inject a sort into the plan to make sure that, uh, when you go, when you go spool data in that data is reused, uh, as often as possible, right? Cause if you have like numbers one through 10 and you have 10 of each, it makes a lot of sense to order those from one to 10, like one, one, one, one, one, two, two, two, two, like on so on. So you get the one, you go look for the one, and then you can reuse the one nine more times. Then you get the two, go get the data for the two, and you can reuse the data for two, nine more times. Uh, so that’s why spools typically show a lot of, a lot more rows coming out of them than going in or something like that. Let’s see. Zane says, you should get both as data flows, then use a merge, and then do conditional splitting. That’s likely your best SSIS flow.

Hit up a Q&A on dba.se, and I’ll help. Zane is always helpful. Zane is one of the most helpful human beings around. Uh, and it’s, it’s, it’s, that’s, that is, he’s right. That is a good question for, uh, dba.stackexchange.com, where you can go and add a lot more detail and, uh, put a lot more, like, you know, give us, give us some, uh, you know, what do you call it there? Uh, screenshots. Everyone loves screenshots. Peter says, my rule for SSIS is to get data from A to B and leave logic outside the packages and solution. Uh, yeah, so I, I do tend to agree there. Um, most specifically because, um, when, uh, I want people to use SSIS, it’s usually in place of linked server queries.

And people will always do this awful thing where they’re like, I’ve got a linked server. I’m just going to write a query. I’m going to write this fancy query, and I’m going to send like a big join condition where clause, something like that out across the linked server query. And it just never tends to end terribly well. There’s also this nasty downside of linked server inserts where I believe they’re row by row. So I usually just like to grab as much stuff, uh, like they’re row by row. If you go from like one out, uh, when you’re bringing a bunch of data in, you can just dump stuff into a table, the query it locally and you’re in much better shape. So, uh, that’s when, that’s what, so when I want people to stop using linked server queries, I usually suggest that they use SSIS instead because it is much, much less sloppy to, you know, go get data, especially if like, you know, have SSIS off on another server, go grab, have it sit up there with its own resources and everything, go grab data, push it around, push it around, push it around. It’s nice. Nice way to do things.

Not that I’ve ever done it, but I hear it’s very nice. SSIS is an ETL tool. So why would you avoid transformation? Oh boy. Religious, getting religious in here. Yeah, I don’t have, I don’t, I don’t know. Don’t ask, don’t ask me that.

Hugo’s on your case now that you better watch out. I’ll be real careful. Hugo, Hugo won’t, Hugo won’t let go. We blogging about you. He will give you the what phone on that. Let’s see. Do we have any email questions coming in? I have a thank you email from SQL bits for attending. Yeah, you’re welcome. SQL bits. That was a great time. I will always go to SQL bits as long as they have me. It’s a, it’s a fun conference, especially because I feel like it’s a bit less stuffy than other conferences. You know, there’s a, they just do such a nice job of making it a fun and, and very friendly environment. And every, every year I see that they, they put these big, big pillows on the ground that people can just hang out in. And every year I see at least like two or three people for the course of the event, just like face down sleeping on pillows. Like, like, like with their luggage next to them or like wearing a backpack. It’s like later. It’s amazing. I love it. Did you go sightseeing anywhere? Uh, so I spent a lot of my time at, uh, at SQL bits with, uh, penal daway. Uh, and that was, that was a lot of fun. We went out to dinner a bunch. Uh, I also, also hung out with, uh, Andy Mallon and, uh, Randolph and, uh, our friend Joanna was a very good time. Uh, sightseeing.

No, not really. Uh, I’m not, I’m not much of a sightseer, especially, uh, my, my idea of sightseeing is to, uh, look at menus and, and, and like wine lists. That’s, that’s my sightseeing. There’s not a lot, no, there’s not a lot of sightseeing in Manchester. I don’t think at least where I was. Uh, it was, it was, it was funny being there because, uh, every, it seems like every trip I take, I have some piece of electronic equipment that just stops working immediately.

Last year it was, uh, my laptop power supply. I plugged it in at, at the first day of our pre-con, I plugged it in to the thing and, uh, plugged it into my laptop and it started leaking. Like it just, it wouldn’t charge. And like this weird fluid, there was like, like, like warm Vaseline just started leaking, like just like grease started cooking out, like cooking oil. So I leaking out of the, out of like the weird rectangle thing that’s in the middle of every, uh, every, every laptop power supply. So that was, that was, that was the year before last. And this year, uh, I had my travel power adapter.

I plugged it in and it made a pop noise and that was the end of it. Uh, nothing would charge in there. So I had to throw that out and it took me a long time to find a new one. I don’t, I walked around for like 45 minutes. I went to like every Sainsbury’s and like electronic store that was around the hotel. And finally I walked into like a random pharmacy, you know, it’s like, do you sell power supplies? I was like, yeah, look at this one. I was like, perfect. I’ll take that. And that wouldn’t work the entire time. It was great.

Uh, let’s see. Didn’t you tweet a picture of a literal, a literal sewage? Yes, that was a literal sewage canal. Uh, well, it was, I mean, when the canal was full of water, it looks much nicer. But the first night I was there, uh, I was, I was, I was walking around at night. I was walking back to the hotel and, uh, I noticed that the canal was drained and it just looked like there was just like, like, like a, like a, like a, like garbage alley the whole way through. Uh, so I don’t know. Max says, did Freddie end up making an appearance?

Why, why didn’t you watch my, my video yet, Max? It’s available on the SQL bits website. You can see for yourself. You can see me dressed up as Freddie with your own eyes. Let’s see. A lot of talk about SSIS over in chat. I’ve seen it correctly version controlled and TFS, but it only lasted until I let one of my colleagues get his mitts on it. I’ve had tons of success doing source controlled SSIS packages. I’ve even built a few semi applications with SSIS. Wow. Holy smokes. You people do a lot with SSIS. I’m glad someone out there does because it’s not going to be me. I’m too old to learn these new tricks.

Far too old. I got to stick, I got to stick to boring stuff like the optimizer. TFS is great for source controlling SSIS and SSRS. That’s the first positive thing I’ve heard about TFS. Usually when people talk about TFS, there’s a long, long string of curse words either before or after, or like even, even in between the T and the F and the S. Usually, usually the F and the S get replaced with some of the things that are not so nice.

Not so family friendly. But yeah, it’s funny to hear that. TFS is good for something. Finally. Thank you, TFS. What a wild ride. Let’s see. Any questions coming in on Twitter? No, not really. All right. Yeah, Manchester was a, was a really good time. I went to, there’s a, there’s a whiskey bar near the hotel called Britain Britain’s Protection, which was very, very good. They had 300 whiskeys. And I think we drank all of them at one point or another. It was a lot of, it’s a lot of that. And then a few nice restaurants. I had, I had a burger in, in Manchester that was not disappointing. Usually, usually in England, the burgers are not like Americans, like up to American standards for burgers, but this one was damn good. That was it. I was, I quite enjoyed that. It was at a, it was at a place called Almost Famous. It’s very good. I would, I would have that burger again.

Someone keeps calling me. I’m not going to talk to them. No, thank you. I like that T-Mobile now tells me when there’s a, when there’s a scam likely. Martin says, Red’s does good burgers in Manchester. Yeah. I heard that Red’s is actually a really, really good barbecue spot. But I just didn’t get a chance to go there. There was a, there was a lot of, I was hanging out with Penal most of the time and he’s a vegetarian. So bringing him to a barbecue place would have been kind of rude. Like here, have like, have some sweet potato.

I got you some broccoli. But, uh, we went to, uh, I don’t know, went to a French place. We went to a Thai place. Everything was generally pretty good. Yeah, it’s right. I mean, if you’re getting a call, it’s probably, probably is a scam. Most likely is most likely is the only people who call me when it’s not a scammer to like, tell me someone is dead or got arrested or something. So I’m like, I will like never pick up my phone. It’s like never good news. It’s always like someone, someone needs money for something. Goodbye. There’s some good mock meat barbecue places in London. Uh, yeah, but we were in Manchester. So, you know, the fake meat was what only what was locally available, unfortunately.

Have I talked to you about your extended warranty? Uh, man, I need, I need one of those on myself. I need an extended warranty on myself. I got life insurance, but I need an extended warranty. Anyway, uh, Martin says, so real meat. Yeah. Real meat. It was, it was all mostly real as far as I could tell. Uh, I would commute four hours to be vegetarian as long as there was a significant amount of alcohol during that commute. You would have to, you would have to really, really get me lit to make that four hour trip worth it. If you’re like, like, like, like if you want me to go four hours for peas, man, you, you would, you would have to, you would have to throw down for that.

Uh, isn’t that called health and extended warranty and yourself, isn’t that called health insurance? Uh, yeah, I guess so. That’s what costs me a lot of money every month. Now you used to cost a whole lot less when I had a real job or, you know, when I had like a, a slightly more real job, it was my other, my, my last fake job had good insurance. Now that I’m paying for it on my own, I have expensive with my, with myself, myself fake job. I have, I have, it’s expected to be inexpensive.

So yeah. So Zane would have to get on the meat train to, for four hours to be a vegetarian. You, you, you, you sort that out yourself. Uh, I think Northern trains are pretty heavily look it up as a rule. Uh, I didn’t get on a train at all there. I took a cab to and from the airport.

I took a plane out and the rest of the time I walked around, I don’t even think I took an Uber or anything like, uh, within the city. I walked everywhere. I was, I was, I was up and down Dean’s gate so much that people probably thought I was a prostitute showing my legs off on the street, all drunk. There was one night I took a detour off Dean’s gate and I walked through like, like shady Jack the Ripper style alleys. I walked by a casino and I walked by a parking garage and there was like a gang of kids drinking beer in the parking garage. And I was like, man, you, you are truly delinquents. If you’re drinking beer in a parking garage like that, you are truly, truly delinquent. You can always decide to migrate to the civilized world. Uh, civilized world wouldn’t have me Hugo. I wish, I wish, I wish they would sometimes sort of, if they would, I would move to, uh, is what my, my, my question, like whenever, whenever I am like, man, what do I want to do for work? My first stop is I look at like French job boards for any vineyards that need a DBA. Unfortunately, most vineyards are not too heavily reliant on SQL Server.

So if I, if I could find a vineyard that needed a DBA, you can, you can bet I’d be out of here. Like pay me whatever you want. Y’all don’t need money. Go hang out. You can, you could, you could almost pay my salary and mine at that point. I would be a okay. Uh, so it’s like, like vineyards and Scottish distilleries. If you could, if like, like, like, like a Freud or Lagavulin or something needed a DBA, I would be out. Like goodbye. Peter says, sounds like you should work with Chris.

Yeah. I would love to, except I don’t know anything about PowerShell, but maybe if she needed a DBA, then, then maybe, but, uh, she could take the PowerShell. I’ll take the SQL Server. It could, could trade that task off. Hugo says, I’d beat you to it. Yeah, you probably would. You’re much closer.

You have a much easier walk, much easier walk than me. Me, I’d be very slow at that. Very, very slow. So, but anyway, I don’t know. Those are, those would be my, my, my, like what I would, what I would leave America for is if LaFroig, Lagavulin, anyone in Chateauneuf-du-Pape, uh, if you’re listening and you need someone to work on SQL Server for you, call me. I’m mostly available. Free 99 for you.

Lovely, lovely people. Lovely, lovely people out there. All right. It’s been a half hour. Uh, I need to go blow my nose and take some like Afrin or something. And, uh, it was lovely talking to you. Hopefully we’ll all be here next week and we can do the same thing. Thanks for, thanks for coming. Thanks for watching. And, uh, I will see you next time. Goodbye.

Going Further


If this is the kind of SQL Server stuff you love learning about, you’ll love my training. Blog readers get 25% off the Everything Bundle — over 100 hours of performance tuning content. Need hands-on help? I offer consulting engagements from targeted investigations to ongoing retainers. Want a quick sanity check before committing to a full engagement? Schedule a call — no commitment required.

Last Week’s Almost Definitely Not Office Hours: March 29

ICYMI


Well, I missed it too. Darn that silly work.

Catch you next time!

Going Further


If this is the kind of SQL Server stuff you love learning about, you’ll love my training. Blog readers get 25% off the Everything Bundle — over 100 hours of performance tuning content. Need hands-on help? I offer consulting engagements from targeted investigations to ongoing retainers. Want a quick sanity check before committing to a full engagement? Schedule a call — no commitment required.

Last Week’s Almost Definitely Not Office Hours: March 22

ICYMI


Last week’s thrilling, stunning, flawless episode of whatever-you-wanna-call-it.

Video Summary

In this video, I found myself live-streaming SQL Server tips and tricks, only to be greeted by the emptiness of an empty chat. It’s moments like these that make you question your entire existence as a content creator. Despite the initial disappointment, we managed to gather a small but enthusiastic audience, including some familiar faces and a few curious newcomers. The discussion ranged from enterprise edition licensing headaches to optimizing SQL Server performance with core-based configurations and max degree of parallelism settings. I also shared insights on free monitoring tools like OpsServer, which might just be the lifesaver for those stretched thin in terms of budget but not in need. The stream turned into a lively debate about workload replay tools and memory allocation strategies, making it an engaging session despite the initial lull.

Full Transcript

I’m live. Uh, no one yet. I don’t even have a thumbs up. I didn’t even give myself a thumbs up. What a sad day. Oh, boy.

So iffy. Will anyone actually be here? How long will I have to wait? How long? One person. One person. There we go. You better have a lot of SQL Server questions, one person, or else I’m screwed. My big plan after this is to go wine shopping for the weekend.

Oh, well, if it’s Josh, I’m not too excited. That’s never a good sign. These developers show up and get nothing out of them. Three of you.

Wonderful. Wonderful. We have enough people to successfully implement quorum until a fourth person shows up. No, there you are. There you are. You ruined the quorum, fourth person. Thanks for nothing. Thanks for nothing.

Do, do, do, do, do, do, do, do, do, do. Do, do, do, do, do, do. Back down to three.

Thank God. now we’re back to quorum now we can successfully run our failover cluster we can successfully fail that was funny it was fun successfully fail fail upward onward and upward that’s my that’s my that’s my plan yeah what do you what do you what do you know about failure kenneth what do you know about it by the way this week’s whatever you want to call it is brought to you by a thing that helps babies fart that’s our sponsor this week so everyone round of applause for our generous sponsor plastic thing that helps babies fart this has never been used by the way just something that came with a new baby a while ago that i found and i thought it looked funny you kind of like a little flute but you know don’t worry that’s never been anywhere weird i still have no idea what that means don’t i don’t want to know i have nothing wow an actual sql server question hello steve how are you uh lars says i use sql enterprise recently came across a 20 cpu limit due to the wrong install media attempts at addition upgrade failed what’s the difference between variants of enterprise edition uh so there is the old version which is like the cal based what you want to do is uh in download the install media that has the core based licensing in the title and do a skew upgrade to that you don’t need to do an edition upgrade you need to do a skew upgrade so um yeah that’s that’s that’s where you want to go with that you need to upgrade additions your edition is fine enterprise is enterprise but the uh the skew of enterprise is what’s skewed up zane says what kind of deluxe baby store did you pick yours up at that it came with accessories uh the hospital where um it it plopped out uh dev dba asks can i send you my copy of great post eric for an autograph uh hmm no no um if you no i don’t even have any copies of that left uh no nothing personal i just don’t want anyone to have my my home address who doesn’t who i don’t know don’t take it personally but everything i do everything from home and it would be weird if people just knew my address but if i if you ever had a conference and i’m at or a SQL saturday or anything that i’m at then i would happily sign it for you for free p.o box yeah no why would i do that yeah like madison yeah madison wisconsin i’m gonna be there what is it friday the 5th for a full day of training and then saturday the 6th where i have a session about the optimizer and how the optimizer works it’ll be fun devdb i don’t know i don’t know what which conferences you get to so be prepared i just might show up anywhere who knows who knows i don’t think i’m going to pass this year though they didn’t pick me for a pre-con and that’s an expensive trip when you’re not getting paid for it expensive if i lived in seattle that’d be one thing yeah bring it everywhere i mean what’s the what’s the worst thing that can happen you’ll run into lots of interesting new people who are like man that’s a cool book what’s that about and then more people will buy buy the book and then i’ll i’ll make another six dollars and that’ll be that’ll be nice yeah mcdonald’s the mall uh rest stop bathrooms you know various basements uh you know bars bars with no one in them that open at 9 00 a.m good places to go good places to be let’s see if there’s any questions uh on another thing yeah no all right good there was a question that came in via email that uh i answered via email and it was it was a it was that someone um someone uses uh sentry one at work but they have so many sql servers that it was getting expensive to keep licensing them all for sentry one because sentry one you know well i mean not just sentry one any any paid monitoring tool is going to be about the same price so it doesn’t matter if it’s like sentry one or uh like quest spot fog or whatever they’re calling it these days but uh yeah like anything’s gonna be about the same price but because there were so many people there are so many servers right there not so many people so many servers who um they needed to monitor uh they were looking for something free or open source and i suggested uh uh so stack overflow and the the very very smart people at stack overflow um have developed their own monitoring tool over the years the price you pay is that it is open source and it’s a little difficult to install but it’s it’s called ops server and it gets a lot of cool stuff it collects a lot of awesome metrics i’m actually gonna pull the link up for it but yeah it’s pretty sweet uh if you can get into if you can if you can get past the the install and config and get it up and running it’s it’s pretty awesome so if you need a free monitoring tool i would do that uh what uh that uh what is that it doesn’t make sense nothing makes sense anymore yes you’re welcome that is a great link it is a wonderful link uh okay i’ll show that uh sql watch dot io wow i need to get a dot io website those things are very popular right what would i do could make it i could get like taxes dot io that would be funny or i get like slow io what else could i do yeah i don’t know lots of stuff dot io that would be a funny url uh have you used the newest azure data studio it has some pretty cool monitoring built in that you can report off of uh no i i have never used azure data studio um because the majority of the stuff that i care about is in execution plans and azure d azure data studio is more like azure duty studio when it comes to execution plans it does not have uh anything anything good going on about it so no and i i i despite a lot of people being very excited about notebooks uh you know i i have i have my own little little pad of paper that i much prefer to keep my notes on my wife’s dad was it was a cop for a long time and when he retired he gave me all his all his like uh his police notebooks so i keep all my notes kind of kind of funny i keep all my notes in like a police notebook and the the first the first the first page has all sorts of funny stuff an ambulance rescue squad fire so like all this like a justice of the peace i don’t know what a justice of the peace does what do they do it’s like it’s like all it’s missing is like a constable in a priest i don’t know what goes on there let’s see here someone has a very important sql server question but they haven’t asked it yet so i’m just going to wait on that i’ll wait 20 minutes if i have to kapil says there was one i recall from spaghetti dba a sql workload yeah if so that’s not i don’t as far as i know that’s not a monitoring tool that allows you to replay a workload on one server from another server in a little bit better way than uh the the built-in tooling uh does that so um that’s that’s all i know about it though that’s what i know that is that is most i know no collusion wants to know how can i get over a million application locks per second the most i can get is 325 000 per second uh try harder that’s what i would do try harder doesn’t sound like you’re trying very hard so eat right get eight hours of sleep and literally just try harder is all you need lar says what do you like to use to replay workloads uh i don’t actually do a lot of that i just don’t um as far as stuff that i know that does it uh what was it benchmark factory is good but expensive um so this used to come up on on other office hours quite a bit where someone want to know how to do that and tara would always talk about her experience where she’s worked at like very large companies uh that had like entire teams dedicated to uh setting up workload tests like they would like they had their own in-house homegrown software uh and they would set stuff up to run to do like have like a like a like a production quality workload inserts updates the leads stuff like that uh yeah distributed replay is a piece of crap it is not fun or funny or useful or amusing it’s like it’s like wow what a great idea imagine being able to do that then you go to use and you’re like hell no i don’t want it i wouldn’t want anything to do with this this is awful dan says if you have a 56 core cpu on a 2012 standard instance of sql server can that cause issues with performance no uh i would so i would say no it’s not going to cause issues with performance standard edition certainly isn’t going to see all those cores just because of the of the limits imposed on it um you know 2017 you can see 24 cores so i the the next question i have is that uh is that um 56 physical cores or is that 56 logical cores with hyperthreading that’s another question who knows let’s see do i have questions coming in from anywhere else no not yet good good good everything’s quiet it’s just you i think we’re alone now what about memory allocation to the cores uh sure so that’s why i was asking uh kind of about the setup over there is um if it’s like if it’s a four-way if it’s a well no maybe yes uh so run sp blitz and see if you have memory offline uh it’ll it’ll warn you about that so uh sp blitz will warn you if you have schedulers offline if you have sequels if you have schedulers on your box that’s equal server isn’t using or if memory is allocated to a numino that sql server can’t currently see so yeah i guess like in like a weird situation if you had like a four-way proc but in like every numino had memory attached to it then yeah you could run into some weirdness with that but um run run blitz and it will tell you about it no collusion insists that having 56 cores on standards and might throw off worker thread counts yeah it very well may very well may uh trying to think of some other funny stuff that might happen let’s see i don’t know like i get a parallelism i don’t know like like i’m just trying to think about like like parallelism and stuff like that but i don’t know it’s it’s one of those things that i’d love to test out but i’m just not good i’m not good at like theoretically figuring out if like one like wait if one numino has like three schedulers hooked up like what will happen yeah look at that look at no collusion with the with the smart answers kavil says if i have a four socket 20 core hyper threading enabled total of 80 logical processors what is my best option to set mac stop eight eight eight just leave it at eight eight’s good uh i wouldn’t i wouldn’t go to one around much past that if you if you set it higher than that it doesn’t really like like so like eight really is kind of like a weird sweet spot for mac stop i have never uh seen an entire workload that benefited from a super high max stop i’ve seen like some queries that benefit benefited from higher max stop but overall setting max stop isn’t just about like you know single query performance either setting max stop is about um you know uh making sure that parallelism doesn’t harm concurrency because if you set max stop to like a very high number then you can like the number of like cores and threads that get spun up for a parallel query can increase exponentially you end up with like a crazy amount of threads allocated to parallel queries and that can that is generally considered a bad thing because the more of these you know super parallel queries you have with lots of threads assigned to them the fewer queries you can run overall you’ll run right into thread pool weights so setting max stop isn’t just about oh gee what’s the best setting for you know like general query performance it really is like a it’s a concurrency setting as well and if you’re not setting cost threshold appropriately aside from that you know setting max stop is you know only only half the battle so careful careful with that uh i would i would start with eight though eight is eight is a nice number for max dot eight is eight is a pretty decent uh sweet spot for query performance and it sounds like the number of cores you have uh hopefully hopefully you have adequate concurrency with max stop set to eight but you might have to you might have to chop max stop down to like six or four you might have to raise cost threshold up to like from 50 to like 75 or 100. who knows i don’t know if you if you’d like more specific advice uh i would i would love to love to help you out in a consulting capacity but all i can give you is vague recommendations set max stop to eight set cost rest so for parallelism to 50 and and tune accordingly let’s see here check db can benefit yes check db can benefit from using the number of cores in a single numenode but who does that god uh let’s see zane says some stuff about some things uh no collusion says if you ever have your work spread over too many schedulers and some schedulers are significantly more busy than the overall query time will degrade because some threads will finish work early that’s also true if you have like two cores though because there’s all those uh there’s all those old craig friedman demos where he make you know he like sets up like a two core vm and he makes one core very very busy with like a max stop one query and then runs a parallel query and you can like see the imbalance i think as long as microsoft hasn’t torn his blog down uh you can still find one of the posts where he does that uh so if you if you’re really interested look for like uh craig friedman post about like parallel scans and i think there’s a demo in there of that happening yeah well i hope you’re not look kabil says uh he’s a consultant too well gee i hope i hope you’re not billing anyone for this time careful on that you’re here you are seen sir you are seen cost threshold for parallelism i start at 50. yes that’s a good starting spot um but it’s it’s funny how that like it’s it’s it’s also it’s also amusing to me how it’s like you have cost threshold set to 50 and that certainly prevents like low cost queries from going parallel but then it’s like like you have like some low cost queries it will never break like like 10 and then you have like like the really bad queries that i’ll have a cost of like 2000. and so and so you’re sending me a look at this like well well cost threshold of 50 is good like it’s keeping like the really small ones from doing anything weird but the man those those those bad queries are really bad so you gotta be a little careful with those uh let’s see no collusion has oh boy no collusions on a roll today holy cow there are some latches and spin locks use some types of queries that don’t scale well with max dot yes it would be nice if someone blogged about those who’s smart and who knows these things and has 96 core servers or or if they’re not haven’t been downgraded to 48 core servers who can who could write about these things with some authority that would be wonderful nesting transaction full is used for parallel inserts into heaps and column stores a good example going above maxed up eight may not help with runtime at all might not you’re right you’re right about that again it would be it would be nice if if someone smart wrote about these things in some detail because the the the only the only other the only person i know who might do that wears a funny hat why should i ever enable auto growth i i enable auto growth all the time can you imagine a job where you you got alerts that you had to go grow a data file i would lose my damn mind and quit auto growth is a wonderful thing auto growth auto growth saves your life just don’t set it to something stupid like a percentage 10 of one gig is not the same as 10 of 100 gigs so i love auto growth i would keep auto growth why not use it for log files i don’t i don’t i don’t know who you’re asking that question to if only conversation threading worked better in here yeah one meg auto growth is a funny one uh that’s one of those one of those cute things that’s like you just well you know the funny thing about the one meg auto growth is at least you’ll never wait a long time for it if you like if you if you were just constantly growing by a meg cool you’re never going to wait a long time to grow by one meg let’s see no collusion says i like auto growth for log files and not data files well aren’t you just a pre-sizing madman oh so okay so then then how how big do you make your data files then if you don’t like them to auto grow that’s what i want to know this is where this is where we see some absurd number like in the in the terabytes i guarantee it you guarantee it how many terabytes as big as required you size queen you dirty dirty size queen how do you how do you know how big is required it’s not that big oh thanks for letting me know that’s the uh lars says he read recently that you might want to allow for enough space to hold two copies of your largest index during rebuilds yeah but who rebuilds indexes that’s for the birds don’t rebuild indexes is a waste of time go do something important like check db or backups no inclusion says i created a database with 72 data files huh how many on purpose like are you sure you didn’t like it wasn’t like a loop that got out of control how big were those 72 data files that’s what i want it must have been huge huge what do you put on 70 what do you put across 72 data files 1.5 so you had i don’t know whatever 70 100 and something gigs i did a little math on the top of my head yeah i want file group that would be funny that would be funny problems with pfs contention yeah that’s a good reason to create that pfs contention is not just for tempdb folks if you are creating enough objects in your data files you can run into page contention in the same way that you can run into it in tempdb and it will show the same exact weight types it’ll show show you those same exact where is it uh the page latch up page latch up weights yeah show you those in the exact same way and there’s no real differentiation like sp who is active can show you like uh which database are happening in but uh if you’re just looking at weight stats and you’re like holy crap i have these page latch up weights what’s going on i already have tempdb set up right could be happening in a regular data file could be anywhere just don’t you could you could just take a a cautionary tale from no collusion and not create well like millions of tables in your user databases or in a short span of time you might run into some scalability issues rcsi is a magic bullet not for pfs contention it’s a magic it’s a magic bullet for uh readers and writers not arguing with each other but not unfortunately for pfs contention that’s not going to help too much uh microsoft is doing some cool stuff with uh with tempdb’s system tables being in memory coming up and it’s in sql server 2019 so that’ll be fun it’s a magic bullet for developer sanity yes that is true otherwise otherwise you make developers spend a lot of time writing no lock hints and they could be doing much much more productive things with their time than writing no lock another quick like the question always comes up when i’m talking to people is uh about if if no lock is different from read uncommitted they think like like one like one is somehow magically better than the other like no it’s the same thing just with less typing well i guess a little bit more typing up front a little bit less typing overall yeah no one wants to solve deadlocks deadlocks are the most boring thing in the world to solve except parallel deadlocks parallel deadlocks make the sexiest deadlock graphs they they they look so cool though it’s like amazing that that kind of line work can be done in a tool designed to write queries and manage a sql server instance with but they’re pretty nifty uh and i like those regular deadlocks this is like i took a key lock i took a key lock i took a key lock i took a uh dull dull dull get up kapil says uh any link that can help me in setting up a four node geo cluster or multi subnet cluster you know what’s funny um my friend sean who works at microsoft had a blog post about that but his blog got taken down recently so i had a link but the link doesn’t work anymore so sorry about that uh you’ll you’ll just have to i don’t know read the documentation let’s see uh someone who who got who got very drunk and didn’t have a good time the next morning at SQL bits is saying they came into a job with deadlock priority low everywhere yeah they that would be a that would be a deadlock priority that would be a deadlock issue huh yeah you look great the next day did you ever figure out where that blood came from i take that as a no yeah that was a good time if anyone if anyone needs a a good place to drink in manchester go to the britain’s protection they have 300 kinds of whiskey and uh and and we have drank them all so that was a good time just stay away from the irish whiskey in a green with a green label because that’ll that’ll do you in yeah that’s where we were yeah good question yeah that was like there’s like a bar within walking distance from the hotel and you didn’t know that’s where we were you wouldn’t you would did what did you like you and martin like bump like bump heads real hard and knock each other out maybe let’s see uh yes where else would we have been that could that could explain it yes that could explain the blood blood can come from many mysterious places mostly inside things though inside living things you could just keep a jar of blood with you sprinkle it on stuff where did it come from i don’t know all right all right any other questions anything else fun going on what’s everyone doing it’s friday some of you should be doing things that are better than this it’s at least it’s 5 30 where you are see darren says last week or week before you said you were doing some century one mining scripts are those still forthcoming yes i actually just scheduled that blog post last night uh that’ll get published you know let me see here uh i’ll go look i have magical powers where i can see when blog posts are gonna happen unfortunately i think i’ll my website is slow who do i call if only there were some kind of website performance tuning expert i could talk to about why my website is slow by the way that that’s just that’s just one query that i have lined up it’s nothing i don’t have like a whole series of them it was just one that i was particularly proud of that’s just one of the things that i’m not going to be able to do it i blame wordpress kenneth uh stephen cutter says any good tips for moving file stream data yes put it in the garbage stop using file stream an awful awful awful feature that is hi devdba says uh trying to get toad to recognize my new tns name dot aura file bad time yeah a lot of things with oracle are a bad time sorry about that uh use sql developer instead i hear jeff smith says that that’s much much better uh let’s see wordpress unshared hosting uh do i have that i don’t know what i have managed maybe i don’t know how do i get a how do i get a faster website who do i pay for like enterprise edition of wordpress zane says oracle sql developer is my nightmare well you should open a ticket about that you know what you should tell jeff smith on twitter that you don’t like don’t like it let’s see here i gotta answer that question though where are my posts where are my posts is uh they’re still loading so i blogged so much i made wordpress slow zane just doesn’t like oracle zane has oracle problems let’s see oh yes so the query that uh i wrote for century one will get published april 10th sorry about that darren uh you know what tell you know what i how about this just because darren because you asked i will have that go out uh when can i get rid of this thing what’s tomorrow the 23rd i have blog posts scheduled until april 12th that’s how crazy my life is that’s how little i have going on with myself so let’s see today’s the 22nd uh and that was uh uh yeah you know what i’ll uh i’ll do i’ll do a quick thing i’ll do a quick schedule change and uh when when i get off here and i’ll have the the century one helper query go out on uh i guess monday of next week so watch watch for it monday the 25th that’s what i’ll do for you because we’re best we’re best friends now and i don’t mind changing my blog post schedule for you let’s see uh lara says not small table with old data but typically only last two weeks of data queried i’ve heard that partitioning for performance is not a good idea but need to keep stats updated to retain good query plans uh so that’s where something like filtered statistics or filtered indexes would probably be helpful uh where so like even so so here’s the thing is even if like you partition the table you’re right it wouldn’t be helpful for performance and it also wouldn’t help stats because while sql server will sample statistics at the partition level the final histogram that it puts together is the same 200 steps for the entire table so it’s not particularly helpful uh for that so it won’t help you any that won’t help you any better like you could update stats for a single partition but it’s not good it’s still going to be the same it’s still going to do the whole 200 table things they’re not gonna i’m not gonna help you with that you really need to do is create like filtered stats or uh filtered indexes that will have filter statistics on your hot data and that way you can figure that way you can keep that chunk of data um you know better stats up to date it did so there’s always that dean says uh dude has a bug that caused it to get stuck when opening okay i don’t want to read that uh darren’s yep no problem darren tana says beats me i only have them uh i don’t know what that means uh yes spiffles let’s see uh did a 3000 word blog post the last day i’m taking a week off what would ever possess you to put 3000 words in one blog post that’s a book that’s like five blog you could have had you could have had like a week of blogs with that why would you put that all in one blog post there’s nothing nothing good about a 3000 no one’s going to get to the end of that no one’s going to get to the end people are going to die reading that thing it’s like a farewell to arms or remembrance of things past it’s far too long to exist sounds like half of a paul white blog post paul thanks to my expert tutelage has gotten much better about breaking his blog posts up into consumable chunks you have me to thank for that that rogo’s blog post was one blog post said four and give it recaps tldr you need that stuff no one’s as smart as you yes that that likely is where the blood came from banging banging your head against a laptop for 3000 words i would do the same thing yeah i think i think rogo’s rogo’s was originally one post it’s like you are out of your damn mind what software do you use to compose blog posts uh google chrome on wordpress uh i don’t use the block editor uh and i don’t use what’s that god i don’t eat gutenberg that thing is such a piece of crap writing blog posts in gutenberg maybe you want to stop blogging i eventually like there was a plug-in to use the classic editor and i’ve never looked back god gutenberg such garbage uh hang on a second my wife is texting me okay all right apparently my wife is ready to go wine shopping so i’m going to get out of here it was lovely having you all this week uh i will see you next week and we will talk about more stuff and things for sql server or whatever you bring up i will maybe maybe i’ll start reviewing wine on here because that would be the only reasonable use of a friday is to drink wine in front of all you people anyway uh thank you all for coming i’ll see you next time goodbye uh remember this week’s sponsor is a thing that helps babies fart so thank you thing that helps babies fart is is is is is is is is

Going Further


If this is the kind of SQL Server stuff you love learning about, you’ll love my training. Blog readers get 25% off the Everything Bundle — over 100 hours of performance tuning content. Need hands-on help? I offer consulting engagements from targeted investigations to ongoing retainers. Want a quick sanity check before committing to a full engagement? Schedule a call — no commitment required.