Get AI-Ready With Erik: Generating Embeddings
Summary
In this video, I delve into the intricacies of generating embeddings in SQL Server 2025, Azure SQL Database, and Azure Managed Instance using T-SQL’s AI generate embeddings function. However, I caution viewers against performing these operations directly within SQL Server due to potential inefficiencies and external dependency issues. The demonstration involves creating a `users_to_embed` table with an ID column and a vector data type embedding column, illustrating the process of generating embeddings for user profiles that include their “About Me” descriptions. While this method works, it highlights the limitations and challenges of using SQL Server for such tasks, suggesting that Python might be a more suitable environment for handling these operations efficiently.
Chapters
- *00:00:00* – Introduction to AI Generate Embeddings in SQL Server
- *00:03:15* – Generating Embeddings for User Profiles
- *00:07:24* – Finding Similar Users Using Vector Distance
- *00:10:10* – Best Practices and Considerations
- *00:11:43* – Conclusion and Next Steps
Full Transcript
Erik Darling here with Darling Data. And in today’s video, we’re going to talk a bit more about generating embeddings in SQL Server 2025, Azure SQL Database, and Azure Managed Instance. To do so, we are going to continue to invoke our T-SQL friend AI generate embeddings. But this time, what I want to do is kind of show you why you might just want to stick with Python on this. Do it outside of SQL Server for the love of God. So, I’m actually not going to drop in, recreate this table because I’ve already got it loaded up and you will, in short order, see why I did not decide to do this all from scratch.
We’d be wasting some time. So, this would be our sort of demonstration table, right? We would have all the things that we would need to be AI ready, like yours truly.
We would have an ID column, which is an integer, right? Because if we wanted to be forward thinking about someday having a vector index on this table, if they ever become not horrible, if they ever have to ever finish that damn thing, instead of like the Homer Simpson car features that we have now. So, we would want to have that up there, but who knows if that restriction will be relaxed when the vector indexes stop being horrible, right?
Like, maybe we’ll be able to use a big int. Like, hmm, you got fabric. Anyway, this is the table we would use and we would have our wonderful embedding column here, which is a vector data type that accepts a 1024 exactly dimension vector and is currently using the float32 model of storage rather than the float16 preview model of storage.
Now, in real life, you know, if I were running through this, I would want to stick maybe about 50 users into the table who have an about me column, which is a reasonable length, right? Because we’re going to be using the about me column for this one, right? For various reasons.
So, like, let’s say, you know, like, there’s lots of stuff in the Stack Overflow database that one could wish to embed, right? We’ve talked about sort of like the post table where you have titles and then you have bodies and, you know, stuff like tags, you know, you probably wouldn’t want to have, you probably wouldn’t want to do that. But then, like, you know, in the comments table, there’s the comment text column and, oh, screw comments.
But then, like, in the users table, you know, like, location, probably not, website, probably not. But about me would actually make a pretty reasonable embedding because what you could do is take the things that someone has written about themselves, right? The kind view that someone has taken on their Stack Overflow profile.
And it’s like, I am the best coder in the world. And you could search for people who might be good at, I don’t know, answering a question or maybe like even, you know, that Stack Overflow job board, which saved the company, thankfully. You know, hey, thank God we have that job board.
And, you know, you could even find maybe people who would be good matches for a particular job if you were to embed their About Me profile and, you know, do some semantic searching on it. Because you could do some similarity search between things in the job posting and what people are out there in the world. Right?
It’s like everything except the salary. But we could stick a bunch of users in here and we could, you know, just to keep, like, the embedding stuff reasonable. We would say that, you know, we want people who have an About Me that is somewhere between, like, one hundred and two thousand characters.
Beyond that, like, what the hell are you rambling on about? It’s like, let me just, like, paste, like, book verses in there. Out of your mind.
So we’ve got this users to embed table. And this users to embed table. And I realize that this sounds a little, you know, after dark. But trust me, it’s really just vector stuff.
We’ve got 50 users in here that we wish to embed. And, of course, the sort of, you know, like yesterday, the way that we generate embeddings in T-SQL is by calling the AI generate embeddings function, passing in a string that we wish to vectorize.
Right? In this case, I am a software developer who loves databases. I’ve never met one. Right?
They all seem to hate databases. It sounds like the passion. It’s like, oh, what can I do to this thing today and then complain about later? Right? Like, JSON. Vector stuff.
Yeah. Anyway. If we run this, we will get back our embedding JSON. Which, again, starting at line two and scrolling on down. All these numbers apparently mean that you’re a software developer who loves databases.
And so we get 1,024 of these numbers back. It’s fantastic news for us because our column is a vector 1,024. Isn’t that, isn’t it amazing the way this stuff just lines right up in a demo?
Right? How could it be better? But, so if we wanted to update just one user, right? Just one.
Let’s just show you that it works. Uh, we could set their embedding column to, uh, pass in a column. Right? We could say, we want to generate an embedding for the about me column. And they’ve got put into the user, demo user embeddings table, uh, just for the first user.
Right? So just, just one of them. And we run this. Right? And it’s, again, not the slowest thing in the world, uh, for, for one row.
But trust me, doing, doing the rest of them, uh, you know, you have like, like, I think most people who use SQL Server are pretty like comfortable with the idea that once modifications go past a certain like rows row count, you’ll probably want to batch things up.
You’ll probably want to think about that a little bit earlier if you are generating embeddings with T-SQL and, uh, calling this. This, because remember like, like, like what this thing calls out to is not inside of SQL server.
SQL Server does not generate embeddings. Right? SQL Server does not have any facility within it to generate an embedding with. Okay.
Uh, this is still an external model. You still have to call outside of the database, have something do something, then send it back to the database.
And then you get to do the update. Right? So I’m, I’m pretty lucky here in that, you know, um, like this is a VM and all of the processing that I’m doing on here takes place in the VM.
Right? But like, you know, if you look over here, you know, like we still have to call out to Olama. Olama has to do something and send it back.
Olama has to talk to this other thing because Obama, not Obama, Olama only allows, uh, only as right now only has HTTP requests and the external endpoints require HTTPS. So you need like a separate thing that has HTTPS that can talk to HTTP.
And so like, like right now there’s like steps to take. Right? And if you were using like open AI, you’re talking about like an internet call, right? So you’re talking about like reaching out to the internet.
So like, like probably want to think real carefully about like, like how many rows at a time you’re going to count on for this. Because, um, if I were to update all 50 of these rows at once, it would be much slower.
I’m just, you know, just not patient enough for that. But if we look at what we like, you know, we updated the table with, you know, we’ll see all the stuff today. Um, you know, like, like looking at the embedding, isn’t all that interesting.
So just that like, that’s the number of bytes that ended up in the table, uh, for the embedding. That’s what a thousand and 24 bytes, uh, thousand and 24 vectors, um, you know, measures out to bite wise. Uh, so just, just happened to be for John Skeet.
Hey, John Skeet. How you doing? But then, uh, you know, if we wanted to find similar users or maybe, you know, we’re, we’re a headhunter and we’re like, uh, I got this stack overflow job board, you know, I’m going to go find some nerds.
Uh, you could do this. Right. And you could, you could write a query. That’s like, I want to find a database performance tuning expert and you could do all this stuff. Right.
Right. And you can even call AI generate. I don’t recommend it. Don’t particularly recommend it. Right. Granted, like when you, like, uh, I don’t particularly recommend it. Um, you could even do this where you could generate an embedding directly in here.
Right. You could say, I want to like, rather than, you know, like, like, like pre, um, like, you know, and, and vectorizing this string and turning it into an embedding. You can turn it into an embedding on the fly with AI generate embeddings right in the, uh, like right in the query.
And then you can just say, well, I want to find the vector distance between, you know, the embedding that’s in the table and the string that I’m sending in. Right. You can do this.
Right. You’re, you’re allowed to, but think, think hard about doing things like this. Like this is maybe not how you want to do things, but we can run this query and we can get back, you know, for our, our very small 50 row table. Uh, we can get back some, some people.
Um, I don’t know you, I, I, I’ve talked to this guy before, GBN. Um, he’s, he’s, he’s a hoot. Um, the rest of these people, I don’t know.
Um, so, um, I, I couldn’t tell you if they’re actually database experts, but yeah. Yeah. Yeah. Yeah.
He is, he is actually a satanic sysadmin, I guess. Um, SQL Server developer Moe. All right. Well, you know, um, just choosing the, the first hundred characters of someone’s about me is maybe not always a winner. But anyway, uh, just a little bit about how you can, um, you know, use AI generate embeddings, uh, inside SQL Server.
Again, SQL Server does not generate embeddings on its own. These are all external calls. So think very, very carefully about, uh, how and where you use them in your queries.
Um, you know, if you need to generate a lot of embeddings, I would still probably want to do that external to SQL Server and then put the data in rather than have SQL Server call out generate embeddings and bring it back in. Um, in the context of a query, you know, smallish tables, you can, you can do this on the fly if you want. But, you know, some testing that I’ve done on larger, um, larger queries was, uh, even though this is just like a scalar value, it’s still not a, not a good time.
Anyway, thank you for watching. I hope you enjoyed yourselves. I hope you learned something and I will see you over in tomorrow’s video.
As always, as usual, where we will talk about, well, we’re gonna, we’re gonna, we got two more unnamed tabs up here of things to talk about. So we’ll, we’ll go and do that. Anyway, thank you for watching.
Goodbye. Thank you for watching. Goodbye.
Going Further
If this is the kind of SQL Server stuff you love learning about, you’ll love my training. Blog readers get 25% off the Everything Bundle — over 100 hours of performance tuning content. Need hands-on help? I offer consulting engagements from targeted investigations to ongoing retainers. Want a quick sanity check before committing to a full engagement? Schedule a call — no commitment required.