Search results matching tag 'SQL Server 2012'

Use TPC Database Benchmarks to Save Money

KKline — Mon, 29 Apr 2013 14:47:00 GMT

Last month, I began a series of articles describing database application benchmarking. In the first article, I told you about different ways that you can construct your own database application benchmark. However, you don’t have to reinvent the wheel. The Transaction Processing Council (www.tpc.org) has already created a large number of database benchmarks that are extremely useful and informative.

I also described last month how the TPC provides several different types of benchmark tests. For example the TPC-C and TPC-E benchmarks are extremely useful for measuring transaction throughput. On the other hand, the TPC – H benchmark is useful for measuring business intelligence workloads.

Today, I would like to give you a primer on how to read the benchmark reports that are published by the major database and hardware vendors. You never know when a vendor will publish a new benchmark. There’s no set schedule for them to publish their test findings. Of course, you can always look for new advertisements from many of the vendors. But that’s very imprecise. I prefer to find out if there are new results on my own and so I typically start at http://tpc.org/information/results.asp. There, I’ll check to see if my favorite hardware or database vendors have published any new test results.... read more ...

Many thanks,

-Kevin

- Follow me on Twitter!

- Google Author

Read the New TPC Database Benchmarking Series

KKline — Mon, 22 Apr 2013 18:17:00 GMT

Let's talk about database application benchmarking.

This is a skill set which, in my opinion, is one of the major differentiators between a journeyman-level DBA and a true master of the trade. In this article published in my monthly column at Database Trends & Applications magazine, I'll give you a brief introduction to TPC benchmarks and, in future articles, I'll be telling you how to extract specific pieces of valuable information from the published benchmark results.

But let's get started with an overview … read more.

Many thanks,

-Kevin

- Follow me on Twitter!
- Google Author

SQL Server 2012 Cumulative Update #7 is available!

AaronBertrand — Mon, 15 Apr 2013 12:45:00 GMT

The SQL Server team has released CU #7 for SQL Server 2012 RTM.

KB article: http://support.microsoft.com/kb/2823247
Build # is 11.0.2405
This build has 17 fixes

Relevant for builds 11.0.2100 -> 11.0.2404. Do not attempt to install on SQL Server 2012 SP1 (any build >= 11.0.3000) or any previous version.

PASS Business Analytics Conference (BAC) Recap

KKline — Sun, 14 Apr 2013 14:15:00 GMT

The PASS Business Analytics Conference (PASS BAC) is PASS' first foray into an event that is dedicated to business intelligence, big data, data visualization, and business analytics. And it totally makes sense for PASS to move in this direction, over and above the flagship community work centered on database management and application development. Why? Because business analytics is all about how to apply the data being collected and managed by all of those developers and DBAs. And, at the end of the day, how we use and apply our data is really the nexus of its value. That's what matters to business. You can read the speech from the standing president, Bill Graziano (Twitter | Blog), or watch it online at the PASS website.

The day one highlight, introduced by the SQL Server team's best presenter - Amir Netz (Twitter), is the release of a new BI data visualization tool called Project “GeoFlow” for Excel. GeoFlow is a 3D visualization and storytelling tool that helps you map, explore and interact with both geographic and chronological data for visualizing data which is difficult to identify in traditional 2D tables and charts. With GeoFlow, you can plot up to a million rows of data in 3D on Bing Maps, see data changes over time and share findings through appealing screenshots and cinematic, guided video tours of the data. It's really something you have to see to understand – check out the video demo and screenshots below. You can also download and try it out firsthand today. It’s an entirely new way to experience and share insights – one you’ll probably enjoy. For more information on GeoFlow, check out the Excel team’s blog and visit the BI website.

The highlight for me, aside from connecting with so many friends and colleagues in the exhibit hall at the SQL Sentry booth, was the day 2 keynote address by Dr. Steve Levitt of Freakonomics fame. Freakonomics is both a brilliant blog and the number one business book in America. His insights are well documented in a variety of places, not just in his own channels, but also in places such as TEDtalks. I'm also really enjoying his new website, https://www.freakonomicsexperiments.com/.

Steve presented an outstanding keynote, full of funny anecdotes and insights into the world of data analytics and interpretation. A couple of his comments really resonated with me which are worth repeating. In one story, he pointed out how some of the greatest insights came from corporate data which was collected incidentally or coincidentally. The data that help provide the greatest and most valuable revelations were from data that was basically a corporate afterthought. Another revelation - he's only now starting to make much use of relational databases. He primarily uses spreadsheets, flat files, and the Stata statistical analysis tool. Another insight, which I've known and proselytized as "the Fresh Pair of Eyes" approach, is that it really helps him to gain insights in a problem by knowing as little about the problem as possible. As it turns out, if you know the industry or the challenge at the core of the problem, you make a lot of assumptions that limit your means of interpreting data. By knowing nothing or next to nothing about a particular problem, you can ask the questions that insiders never ask. Here's an example (not from the keynote though) - let's say you're an energy company CEO. You might spend a lot of time thinking about how to accommodate the expected huge increase in energy consumption due to lots of people driving electric cars. You might tell your data analysts to figure out when and how to ensure peak electrical usage is available at the times when consumers are recharging their electric vehicles. But a fresh pair of eyes would point out that electric cars, in their present form, are a huge energy boondoggle compared to hybrid and plain ol' cheap, high-mileage models like the Honda Civic. Consumers will never recoup their investment in a high-priced, all-electric car compared to a cheap, gas sipping model.

At the heart of his presentation is the fact that data is meaningless when it doesn't answer important questions! Many times, data professionals spend so much time devising elegant SQL statements and clever user-interfaces that they forget about using a fresh pair of eyes when they look at business questions. Our session, Operational Excellence for the BI Pro, focused on the trails and travails of successfully implementing and growing the footprint of a business intelligence project.

In addition, we had a fun and very informative panel discussion breakfast on Thursday of the PASS BAC. At right is a picture of Nick Harshbarger, Justin Randal, and me prior to the session. The audience was very engaged and, despite having no slides, there was a whole lot of wisdom goin' on. The panel included Chris Webb (Twitter | Blog), Craig Utley, Jen Stirrup (Twitter | Blog), Paul Turley (Blog), and Stacia Misner (Twitter | Blog). I served as the moderator and facilitator of the session. We recorded the session, with a little HD Flip camera, and although I haven't checked out the file yet, we're hopeful we can post it or at least a transcript soon.

Do you have a "fresh eyes" story? I'd love to hear it! Post a comment here!

Many thanks,

-Kevin

-Follow me on Twitter!
- Google Author

The All-New 'Database Lifecycle Management" is available on MSDN

KKline — Fri, 05 Apr 2013 10:51:00 GMT

The initial release of Database Lifecycle Management is now available on MSDN.

The site is something called "curated content". This means it's a single consolidated location to look up lots of disparate articles and content, all in one easy to search location. This “curated content view” contains the best content, video, and community-centric information from Microsoft, including topics like:

SQL Server Data Tools

· Get started with sample projects, code samples

· Video demos by Gert Drapers (blog)

· Script common data portability tasks using Sqlpackage.exe

· Link to the SSDT team blog

SQL Server Management Studio

· Manage SQL Database using SSMS

· Backup and restore w/ SQL Azure

· Migrate local databases to Azure

· Video demo of hybrid scenarios by Gert Drapers (blog)

Windows Azure SQL Database

· SQL Database backup and restore

· Import/export SQL Database

· Windows Azure training kit

· Connection management and troubleshooting connections

Enjoy,

-Kevin

-Follow me on Twitter!
- Google Author

Squishy Limits in SQL Server Express Edition

KKline — Thu, 28 Mar 2013 12:19:00 GMT

It's an old story you've probably heard before. Provide a free version of your software product with strict limitations on performance or other specific capabilities so that folks can give it a try without risk, while you minimize the chance of cannibalizing sales of your commercial products. Microsoft has take this strategy with SQL Server Express Edition, not only to increase adoption in the student market but also to counter the threat of open-source (i.e. free) relational databases like MySQL for entry-level applications.

One such limitation of SQL Server Express Edition is that it supports no more than 1GB of RAM for the instance. Of course, you could have many Express Edition instances on a single Windows server, each with its own 1GB of RAM.

But what does that metric of 1GB of RAM actually mean? The key thing to remember is that the restriction is for buffer cache. Since SQL Server has many other caches, even when not counting the plan cache, there are plenty of other caches within SQL Server. (Run a query against sys.dm_os_memory_clerks if you'd like to see some of the others). Because only the buffer cache has the strict 1GB limitation, you can actually watch SQL Server Express Edition's memory working set size grow to around 1.4-1.5GB due to the other memory caches at play.

Pawel Potasinski, a SQL Server MVP from Poland (Twitter | Blog), once posted an interesting repro for this behavior:

-- Assess amount of databases resident in buffer cache

SELECT
 CASE
 WHEN database_id = 32767 THEN 'mssqlsystemresource'
 ELSE DB_NAME(database_id)
 END AS [Database],
 CONVERT(numeric(38,2),(8.0 / 1024) * COUNT(*)) AS [MB in buffer cache] 
FROM sys.dm_os_buffer_descriptors 
GROUP BY database_id 
ORDER BY 2 DESC; 
GO

-- Assess amount of tables resident in buffer cache
SELECT
 QUOTENAME(OBJECT_SCHEMA_NAME(p.object_id)) + '.' +
 QUOTENAME(OBJECT_NAME(p.object_id)) AS [Object],
 CONVERT(numeric(38,2),(8.0 / 1024) * COUNT(*)) AS [MB In buffer cache] 
FROM sys.dm_os_buffer_descriptors AS d 
 INNER JOIN sys.allocation_units AS u ON d.allocation_unit_id = u.allocation_unit_id 
 INNER JOIN sys.partitions AS p ON (u.type IN (1,3) AND u.container_id = p.hobt_id) OR (u.type = 2 AND u.container_id = p.partition_id) 
WHERE d.database_id = DB_ID() 
GROUP BY QUOTENAME(OBJECT_SCHEMA_NAME(p.object_id)) + '.' + QUOTENAME(OBJECT_NAME(p.object_id))
ORDER BY [Object] DESC;
GO

-- Fill up Express Edition's buffer allocation
IF OBJECT_ID(N'dbo.test', N'U') IS NOT NULL
 DROP TABLE dbo.test;
GO

CREATE TABLE dbo.test (col_a char(8000));
GO

INSERT INTO dbo.test (col_a)
 SELECT REPLICATE('col_a', 8000)
 FROM sys.all_objects 
 WHERE is_ms_shipped = 1;

CHECKPOINT; 
GO 100

The bottom line for the hard memory limit of SQL Server Express Edition is "Yes, it's limited. But it's a squishy limit. Not a hard limit."

Although your mileage may vary, I'd bet a dollar that you'll find more than 1GB in the active working set for your instance of SQL Server Express Edition. I am curious, however, if you're seeing much variation between versions and even service packs of SQL Server? Let me know if you try this out on more than one version and/or service pack level of SQL Server. Did it change much between versions? Let me know!

Enjoy,

-Kevin

-Follow me on Twitter!

Google Author

SQLintersection!

KKline — Wed, 27 Mar 2013 13:47:00 GMT

The best emotion to describe how I'm feeling is 'astounded'. I'm astounded that I'm in such august company to be speaking the SQLIntersection (#iSQL) conference. Read the blog post from my first SQL Server mentor, Kimberly Tripp, which tells you all about SQLintersection.

Check out this list of speakers:

Aaron Bertrand, Sr. Consultant, SQL Sentry, Inc. [blog | twitter]
Andrew J. Kelly, Mentor, SolidQ [blog | twitter]
Bob Ward, Principal Architect Escalation Engineer, Microsoft [blog | twitter]
Brent Ozar, Brent Ozar Unlimited [blog | twitter]
Conor Cunningham, Principal Architect, SQL Server, Microsoft [blog]
Grant Fritchey, Product Evangelist, Red Gate Software [blog | twitter]
Jeremiah Peschka, Brent Ozar Unlimited [blog | twitter]
Joseph Sack, Principal Consultant, SQLskills.com [blog | twitter]
Kendra Little, Managing Director, Brent Ozar Unlimited [blog | twitter]
Kevin Kline, Director of Engineering Services, SQL Sentry, Inc. [blog | twitter]
Kimberly L. Tripp, President/Founder, SQLskills.com [blog | twitter]
Mat Young, Senior Director of Products, Fusion-io [blog | twitter]
Paul S. Randal, CEO / Owner, SQLskills.com [blog | twitter]
Paul White, SQL Kiwi Limited [blog | twitter]
Steve Jones, Editor, SQLServerCentral.com [blog | twitter]
Sumeet Bansal, Principal Solutions Architect, Fusion-io [blog | twitter]

Read the list of SQL Server sessions here. On top of the list of outstanding sessions to attend, I'll be giving a keynote on Tuesday afternoon. Witness:

So the only thing between the attendees and the booze in the reception hall is our keynote address?!? Oh yeah, that's going to go down real smooth, isn't it? I'll last about as long as a puny henchman between James Bond and the villain of the movie. Sumeet Bansal, from Fusion-IO, will have to survive until the credits roll. We'll be talking about high performance computing on SQL Server 2012 with an eye towards high availability, AlwaysOn, and Availability Groups.

If you're in Las Vegas, I hope to see you there! If not, you should consider coming to this excellent conference.

-Kevin

-Follow me on Twitter!

Connected development in SSDT versus SSMS

jamiet — Tue, 19 Mar 2013 16:28:00 GMT

When you install the database projects template of SSDT you get SQL Server Object Explorer (SSOX) installed as well. SSOX is a pane within Visual Studio and is the main enabler of the Connected Development experience that the SSDT team have attempted to provide.

SSOX provides some really cool capabilities that are not in SQL Server Management Studio (I hope to blog about them in the near future). In theory these capabilities make it possible for a database developer to spend all their time in SSDT (i.e. Visual Studio) thus making SSMS a pureplay DBA tool (this does of course depend on your definition of both a database developer and a DBA, but I’m not getting into that debate here).

With that in mind I have spent a few days trying to work without SSMS, preferring to live wholly inside Visual Studio instead. By and large I was able to do everything I needed to do from within Visual Studio however there were a few nuances about the experience that kept pushing me back to SSMS, I detail those nuances below.

Server groups

SSOX combines the functions of SSMS’s Object Explorer and Registered Servers pane. I don’t mind either way of working but it does mean that there is no ability to group servers in SSOX like you can in the Registered Servers pane

F6

In SSMS I regularly use the F6 keyboard shortcut to jump between the query, results & messages panes of a query window. No such keyboard shortcut exists in SSDT and they’ve already canned my request on Connect to get this fixed (even though it laughably has status “closed as fixed”).

UPDATE: See the comments below where Brett Gerhardi informed me of a different keyboard shortcut that does the same thing as F6. Actually its not quite the same, if you have multiple resultsets in your results pane then the behaviour is slightly different to F6 in SSMS - but that's not an issue you'll hot frequently.

Change Connection

The context menu in SSMS provides the ability to change a connection as well as connect and disconnect:

SSDT doesn’t have change connection and believe me, you don’t know how much you use a feature until its not there:

There’s also no hotkey to jump to “Connection” on the context menu like there is in SSMS (“C”) and I find that annoying too.

Those were the main annoyances that forced me back to SSMS. The lack of F6 was a major bugbear for me as I am a big keyboard shortcut junkie. If such things don’t bother you then you may be able to live in Visual Studio quite happily. If you have any similar experiences to share I’d be keen to read them.

@Jamiet

March 2013 Cumulative Updates - SQL Server 2012 SP1 & 2008 SP3

AaronBertrand — Mon, 18 Mar 2013 21:40:00 GMT

SQL Server 2012 Service Pack 1 Cumulative Update #3

Build # 11.0.3349
KB Article: KB #2812412
38 fixes!
Relevant for builds 11.0.3000 -> 11.0.3348
NOT for SQL Server 2012 RTM (11.0.2100 -> 11.0.2999)

SQL Server 2008 Service Pack 3 Cumulative Update #10

Build # 10.00.5835
KB Article: KB #2814783
8 fixes
Relevant for builds 10.00.5500 -> 10.00.5834
NOT for SQL Server 2008 R2 (10.50.xxxx)

SQL Server Backup to Cloud – Managing Interrupted backups

SQLOS Team — Tue, 12 Mar 2013 21:48:00 GMT

Writer: Karthika Raman

Technical Reviewers: Guy Bowerman, Pat Schaefer, Andrew Cherry

On January 24, 2013, the SQL Server engineering team released new functionality in SQL Server 2012 SP1 CU2, enabling SQL Server native backup to cloud. The steps to creating a SQL Server backup to Windows Azure Blob storage are simple, but if a backup is interrupted it can leave behind blob files that require additional steps to delete. Some of this information along with other best practices and troubleshooting tips is addressed in the Best Practices topic in SQL Server Books online. In this blog post, we focus on the interrupted backup scenario, identifying locked blobs that might be partial or corrupt as a result of interrupted backups, and deleting them.

The Blob, Blob Leases, and SQL Server section provides background information about blobs and leases in relation to SQL Server Backup to cloud. The Managing Interrupted Backup section discusses managing interrupted backups and a sample PowerShell code you can use to get started.

Blobs, Blob Leases, and SQL Server

In order to get exclusive write access to a blob, a lease is acquired. Acquiring a lease helps avoid accidental overwrites or deletes when there is already another process accessing a blob. Blob leases are two types: One type of lease has a set duration. This duration can be anywhere from 15 to 60 seconds. The lease is active for the time of duration, but can be reacquired before it expires, to complete the backup or restore process, by specifying the existing lease Id. The other type where there is no set duration is an infinite lease. Infinite leases are active unless explicitly released or broken. In effect, the infinite lease on a blob acts like a lock.

SQL Server acquires an infinite lease for backup and restore processes. In the case of backup, a unique lease Id is used, and in the case of restore, a known lease id ‘BAC2BAC2BAC2BAC2BAC2BAC2BAC2BAC2’ is used. Once the backup or restore process is successfully completed, the lease is released.

What happens when a backup or restore is interrupted?

During restores, the restore process always attempts to release the lease unless the network is interrupted. If the lease remains active, overwrites or deletes cannot be performed until the lease is broken or released through a subsequent restore. So an interrupted restore may not often be something that requires close monitoring. However, interrupted backups present a different scenario.

If the backup process is interrupted, it can result in a partial or corrupt blob with an active lease which remains, blocking any overwrites, or deletes. It also prevents any restores using this file, since the restore process needs to acquire a lease with the well-known lease Id, but the existing active backup lease prevents this – which is probably a desirable result as the file could be partial or corrupt. So with an active lease on the blob, the blob remains locked until the lease is explicitly broken.

Managing Interrupted Backups

As stated in the section above, interrupted backups need to be monitored and managed. This section walks through the process of monitoring interrupted backups, identifying locked blobs, and breaking the lease using PowerShell scripts.

Backups can be interrupted due to several reasons such as network failures, process canceled by the user, power outage etc. As interrupted backups can result in a partial blob with an active lease, in order to overwrite this file or delete it, you must first identify such blobs and break the lease.

One way to identify such blobs is to actively monitor backup return codes/errors and interrupted backups. Below is a list of error/return codes that are returned from SQL Server Backup and restore process, which you can use to monitor for blob files with active leases.

Msg 3202, Level 16, State 1, Line 1

Write on "https ://mystorage.blob.core.windows.net/mycontainer/test.bak" failed: Backup to URL received an exception from the remote endpoint. Exception Message: The remote server returned an error: (412) There is currently a lease on the blob and no lease ID was specified in the request.

BackupIoRequest::ReportIoError: read failure on backup device https://mystorage.blob.core.windows.net/mycontainer/test.bak '. Operating system error Backup to URL received an exception from the remote endpoint. Exception Message: The remote server returned an error: (409) Conflict..

This method won’t catch cases where the backup process is aborted or the machine goes down.

Another way is to run a script periodically to look for blobs with active leases in the storage account. The PowerShell script example included here is a good starting point. The PowerShell script establishes an authenticated connection to the storage account, looks for all files in a specified container that are in a ‘locked’ state and filters out restore leases using the well-known lease Id.

Below is the PowerShell example to identify locked blobs and breaking the lease. A downloadable version of the code is available on the TechNet code gallery.

Important Note:

This example is intended to show how to break a lease, but running this while a backup is running will cause the backup to fail as it will break the lease that SQL Server acquired to do the backup. Before running this script or scheduling it, ensure that no backup is running at the same time.

param(

[Parameter(Mandatory=$true)]

[string]$storageAccount,

[Parameter(Mandatory=$true)]

[string]$storageKey,

[Parameter(Mandatory=$true)]

[string]$blobContainer,

[Parameter(Mandatory=$true)]

[string]$storageAssemblyPath

)

# Well known Restore Lease ID

$restoreLeaseId = "BAC2BAC2BAC2BAC2BAC2BAC2BAC2BAC2"

# Load the storage assembly without locking the file for the duration of the PowerShell session

$bytes = [System.IO.File]::ReadAllBytes($storageAssemblyPath)

[System.Reflection.Assembly]::Load($bytes)

$cred = New-Object 'Microsoft.WindowsAzure.Storage.Auth.StorageCredentials' $storageAccount, $storageKey

$client = New-Object 'Microsoft.WindowsAzure.Storage.Blob.CloudBlobClient' "https://$storageAccount.blob.core.windows.net", $cred

$container = $client.GetContainerReference($blobContainer)

#list all the blobs

$allBlobs = $container.ListBlobs()

$lockedBlobs = @()

# filter blobs that are have Lease Status as "locked"

foreach($blob in $allBlobs)

{

$blobProperties = $blob.Properties

if($blobProperties.LeaseStatus -eq "Locked")

{

$lockedBlobs += $blob

}

if ($lockedBlobs.Count -eq 0)

{

Write-Host " There are no blobs with locked lease status"

}

if($lockedBlobs.Count -gt 0)

{

write-host "Breaking leases"

foreach($blob in $lockedBlobs )

{

try

{

$blob.AcquireLease($null, $restoreLeaseId, $null, $null, $null)

Write-Host "The lease on $($blob.Uri) is a restore lease"

}

catch [Microsoft.WindowsAzure.Storage.StorageException]

{

if($_.Exception.RequestInformation.HttpStatusCode -eq 409)

{

Write-Host "The lease on $($blob.Uri) is not a restore lease"

}

Write-Host "Breaking lease on $($blob.Uri)"

$blob.BreakLease($(New-TimeSpan), $null, $null, $null) | Out-Null

}

How to test the interrupted backup scenario using the script.

The following is a self-guided walkthrough of creating a locked blob by interrupting a backup process, and using the script to release the lease to successfully delete the blob file that resulted from the interrupted backup.

The script prompts for storage account name, storage access key, container name, and the path and file name of the Windows Azure Storage Assembly. You can either choose to provide these interactively or you can replace the params () section like this:

#provide values for parameters

$storageAccount = "mycloudstorage"

$storageKey = "<primary/secondary access key value of the storage account>”

$blobContainer = "sqlbackup"

$storageAssemblyPath = "C:\Program Files\Microsoft SQL Server\MSSQL11.MSSQLSERVER\MSSQL\Binn\Microsoft.WindowsAzure.Storage.dll"

Note: The storage assembly is stored in the Binn folder of the SQL Server 2012 SP1 CU2 instance. In the above example, it is the default SQL Server instance.

Create a SQL Server Backup to Windows Azure blob storage which completes successfully.

Run the PowerShell script. It returns a message that says that there are no locked blobs.

Now run another SQL Server backup but stop it before it can complete to simulate interrupted backup scenario.

Check to see if the file was created on the storage account.

Try to overwrite the file – see error:

Try to delete – see error:

Run the PowerShell script again. This time you should see that locked blob is identified, and after checking that it is not a restore lease, the lease is broken.

Try overwriting or deleting the file. It should work!

Here are some ways you can enhance this script to automate this process:

Output the list of all blobs that are locked to a report (ex: excel file).
Break the active leases and report out the list.
Break the active leases and report out the list, but remove the filter for restore lease.
Delete the blobs with active backup leases and report out the list.

NOTE: If you are running PowerShell 2.0, you may have problems loading the Microsoft WindowsAzure.Storage.dll assembly. We recommend that you upgrade to Powershell 3.0 to solve the issue. You may also use the following workaround for PowerShell 2.0:

Create or modify the powershell.exe.config file to load .NET 2.0 and .NET 4.0 assemblies at runtime with the following:

<?xml version="1.0"?>

</startup>

</configuration>

For more information on best practices, see SQL Server Backup and Restore Best Practices (Windows Azure Blob Storage Service)

Originally posted at http://blogs.msdn.com/b/sqlosteam/