Geek City: Space Used By Worktables

Kalen Delaney — Wed, 26 Nov 2008 17:08:00 GMT

Today, a reader asked me the following:

"How can I find the amount of space occupied by a worktable?. Using SET STATISTICS IO ON, I can only see the number of reads using the worktable, not the amount of space taken."

What is a worktable?

I always like to think of it as a temp table that SQL Server builds without being asked. While preparing to write this post, I decided to see if I could find a formal definition. Books Online for SQL Server 2005 gives the following definition in the topic "Worktables":

Worktables are generated for certain GROUP BY, ORDER BY, or UNION queries. For example, if an ORDER BY clause references columns that are not covered by any indexes, the relational engine may need to generate a worktable to sort the result set into the order requested. Worktables are also sometimes used as spools that temporarily hold the result of executing a part of a query plan.

Then I went to one of my favorite whitepapers, "Working with tempdb in SQL Server 2005", which I strongly suggest you take a look at, if you're at all interested in keeping track of your tempdb database. It had a slightly different definition:

Work tables are temporary objects and are used to store results for query spool, LOB variables, and cursors.

So there is some overlap, in that both definitions mention spools.

Prior to SQL Server 2005, the best we could do was watch the STATISTICS IO value, and look at the page reads for any worktables created in the query, but, as my reader mentions, those values show us the number of reads, not the total size of the tables . There were/are some Performance Monitor counters that let us see how many worktables were created, but they don't mention the size.

SQL Server 2005 provides us a couple of DMVs that can be helpful.

The first, sys.dm_db_file_space_usage, has a name that seems like it will provide information about all your databases, but it turns out it just provides information for tempdb. I usually use this view to keep track of the version store space, but it also tells me how much space is used for user objects (explicit temp tables) and internal objects (which include worktables).

The second, sys.dm_db_session_space_usage, reports information for each session, so you can filter it by the session_id you are interested in . For you current session, you can look at @@spid:

SELECT * FROM sys.dm_db_session_space_usage
WHERE session_id = @@spid;

During testing, the above can be useful, to look at the values before you run a test, and then look at the values afterwards, and compute the difference. This still doesn't give you the exact size of your worktables, but it can give you some ideas. In fact, the above mentioned whitepaper states that there is no way to get the number of pages used by any specific internal object in tempdb.

The whitepaper gives you code to create a table called tempdb_space_usage and a stored procedure called sp_sampleTempDbSpaceUsage to populate the table. It also provides half a dozen queries to examine the data collected.

You should be able to get a much better handle on what is using your tempdb space by following the guidelines in the whitepaper and running some of the provided queries.

Have fun!

~Kalen

Did You Know? Estimated vs Actual Plans

Kalen Delaney — Mon, 30 Jul 2007 18:41:00 GMT

In my previous post, I mentioned that it is important to understand the difference between estimated and actual query plans, so I decided to go into a few more details regarding the differences.

Optimization takes place before execution, so in one sense, any query plan is an estimated plan. But when you request an actual plan, SQL Server will actually execute each statement as it displays the plan, so the plan for subsequent statements can change after the earlier statements are executed. In addition, SQL Server adds additional information to the actual plan, after the execution takes place. As mentioned last time, the actual plan includes the number of processors used to execute the query. It will also include the number of rows returned from each step of the plan as it is executed, and the number of times each step is executed. If you are executing a plan that has already been cached and is now being reused, the actual plan will include both the parameter values the plan was originally compiled with and the parameter values used during execution.

You can request that SQL Server display ESTIMATED plans without executing the query with any of the following options:

SET SHOWPLAN_TEXT ON

SET SHOWPLAN_ALL ON

SET SHOWPLAN_XML ON

For graphical estimated plans, you can use the Query | Display Estimated Execution Plan menu option. This can be invoked with a toolbar button, or with Cntl-L.

You can request that SQL Server display actual plans with any of the following options:

SET STATISTICS PROFILE ON

SET STATISTICS XML ON

For graphical actual plans, you can use the Query | Include Actual Execution Plan menu option. This can be invoked with a toolbar button, or with Cntl-K.

I'll show you 3 examples of ways in which the actual plan can be useful. They require that you have the AdventureWorks database installed.

Example 1:

First, create a stored procedure called 'testtemp' that will build a temporary table.

USE AdventureWorks
GO
IF EXISTS (SELECT 1 FROM sys.procedures
WHERE name = 'testtemp')
DROP PROC testtemp
GO
CREATE PROC testtemp
(@p int)
AS
SELECT * INTO #t
FROM Sales.SalesOrderDetail
CREATE INDEX t_index ON #t (ProductID)
SELECT * FROM #t
WHERE ProductID < @p
RETURN
GO

Try to look at the estimated plan for the procedure, and you will get an error because the plan for the SELECT cannot be generated when the temp table #t does not exist:

-- Estimated plan
SET SHOWPLAN_TEXT ON
GO
EXEC testtemp 896
-- error is generated
GO
SET SHOWPLAN_TEXT OFF
GO

Now look at the actual plan and you will see plans for the temp table creation and for the SELECT from the temp table.

SET STATISTICS PROFILE ON
GO
EXEC testtemp 896
GO
SET STATISTICS PROFILE OFF
GO

Example 2:

One of the main reasons that the actual plan may different from the estimated plan is because of data changes to your data. In general (and assuming you have the option 'auto update statistics' enabled) if more than 20% of the data in a table changes, the optimizer will detect stale statistics and update them automatically. The updated statistics will then trigger a recompile.

This example makes a copy of Sales.SalesOrderDetail in the AdventureWorks database and builds an index on the table.

SELECT * INTO NewOrders
FROM Sales.SalesOrderDetail
GO
CREATE INDEX IX_NewOrders_ProductID on NewOrders(ProductID)
GO
SET SHOWPLAN_ALL ON -- Estimated Plan
GO
-- The estimated plan shows us that a seek is done on the SELECT from NewOrders because at optimization time
-- there are only a few rows that have a ProductID value of 897.
BEGIN TRAN
UPDATE NewOrders
SET ProductID = 897
WHERE ProductID between 800 and 900
SELECT OrderQty, CarrierTrackingNumber
FROM NewOrders
WHERE ProductID = 897
ROLLBACK TRAN
GO
SET SHOWPLAN_ALL OFF
GO

-- The actual plan shows us that a table scan is done on the SELECT from NewOrders because after the update is executed,
-- statistics are updated and the query is recompiled. Now there are lots of rows that have a ProductID value of 897.
SET STATISTICS PROFILE ON -- Actual Plan
GO
BEGIN TRAN
UPDATE NewOrders
SET ProductID = 897
WHERE ProductID between 800 and 900

SELECT OrderQty, CarrierTrackingNumber
FROM NewOrders
WHERE ProductID = 897
ROLLBACK TRAN
GO

SET STATISTICS PROFILE OFF
GO

Example 3:

For this example, you can create a stored procedure that uses a parameter to determine which rows from the Sales.SalesOrderDetail table to select.

USE AdventureWorks
GO
IF EXISTS (SELECT 1 FROM sys.procedures
WHERE name = 'GetProducts')
DROP PROC GetProducts
GO
CREATE PROC GetProducts
(@p int)
AS
SELECT *
FROM Sales.SalesOrderDetail
WHERE ProductID = @p
RETURN
GO

The first time you run the procedure, a plan will be built based on the first parameter. The second time you run the procedure, the original plan will be used, and the statistics should show that SQL Server is performing more reads than there are pages in the table. (There are 1238 pages in the table.)

SET STATISTICS IO ON
GO
EXEC GetProducts 710 -- using nc index takes 145 logical reads
GO
EXEC GetProducts 707 -- using nc index takes 9458 logical reads.
GO

You can look at the actual plan in XML to see the parameters used for compile and execute.

SET STATISTICS XML ON
GO
EXEC GetProducts 707
GO

Near the bottom of the XML document, you should see the following:

<ParameterList>

<ColumnReference Column="@p" ParameterCompiledValue="(710)" ParameterRuntimeValue="(707)" />

</ParameterList>

Seeing the different values used for compilation and execution can lead you to suspect an issue with parameter sniffing. There are several ways around this problem, but that's the topic for another day.

Have fun!

~Kalen

Kalen Delaney : temp tables

Geek City: Space Used By Worktables

Did You Know? Estimated vs Actual Plans