Search results matching tag 'backup'

The All-New 'Database Lifecycle Management" is available on MSDN

KKline — Fri, 05 Apr 2013 10:51:00 GMT

The initial release of Database Lifecycle Management is now available on MSDN.

The site is something called "curated content". This means it's a single consolidated location to look up lots of disparate articles and content, all in one easy to search location. This “curated content view” contains the best content, video, and community-centric information from Microsoft, including topics like:

SQL Server Data Tools

· Get started with sample projects, code samples

· Video demos by Gert Drapers (blog)

· Script common data portability tasks using Sqlpackage.exe

· Link to the SSDT team blog

SQL Server Management Studio

· Manage SQL Database using SSMS

· Backup and restore w/ SQL Azure

· Migrate local databases to Azure

· Video demo of hybrid scenarios by Gert Drapers (blog)

Windows Azure SQL Database

· SQL Database backup and restore

· Import/export SQL Database

· Windows Azure training kit

· Connection management and troubleshooting connections

Enjoy,

-Kevin

-Follow me on Twitter!
- Google Author

SQL Server 2008R2 / 2012 Standard & Backup Compression

manowar — Mon, 25 Mar 2013 13:17:06 GMT

This is something that is totally overlooked, in my experience, with SQL Server 2008 R2 and SQL Server 2012. On the contrary to what happens with SQL Server 2008, the Standard version of SQL Server 2008R2 & SQL Server 2012 supports backup compression:

SQL Server 2008 R2 BOL
http://msdn.microsoft.com/en-us/library/cc645993(v=sql.105).aspx
SQL Server 2012 BOL
http://msdn.microsoft.com/en-us/library/cc645993(v=sql.110).aspx

Unfortunately a bug in the documentation said the opposite in the past, but it has been fixed quite a long ago now:

http://social.msdn.microsoft.com/Forums/en-US/sqldocumentation/thread/b4f846e8-a339-422c-bb0b-91751e6c8560/

Just keep it mind

SSMS 2012 Restore GUI Gotcha

merrillaldrich — Fri, 15 Mar 2013 16:30:07 GMT

Today I want to bring to your attention an issue in the SQL Server Management Studio 2012 restore GUI. In many ways the new restore dialog is nicer than the old one, with new features and added convenience – but, as is always the Achilles heel of GUI tools like this, if you don’t know what’s really going on it can bite you. I’m not sure what to call this issue, maybe just a UI design flaw. Technically it works as designed, and there’s nothing really wrong with it, so it’s not a bug. But I can imagine it really causing someone pain who is careless or doesn’t know what’s happening behind the scenes.

Restoring a copy of a database make take the original down.

Among the new features is some management of “Tail-Log Backups,” which is a wonderful idea. Taking a backup of the tail of the log will take down the database and put the whole end of the log into a backup file, preserving basically all modifications to the DB from the log. This is ideal, for example, at the last step of migrating a database using log shipping, because all the activity is preserved and the source database is locked against further modifications. The log chain is preserved in moving to the new copy of the database, or in a DR scenario, to your other server.

The problem is, I know a lot of people don’t know this. Often its people who use the GUI a lot, such as “accidental DBAs.” I think the GUI is really important despite those who rarely use it, or frown on it.

Here’s the issue: Suppose I have a production database that has some user-created problem (like a user accidentally updated some rows.) It may be reasonable to restore a copy of the database to a point before the issue and investigate whether it’s possible to merge the data back in. The original database isn’t damaged from a technical point of view – there’s no corruption, for example.

Right-click the original source database and choose Tasks > Restore > Database … from the context menu.
The resulting dialog will go out and discover the backup files for that database from MSDB, which is very helpful.
Change the name of the target database, in order to restore it as a copy.

There’s a note at the top of the dialog that, while accurate, may not help some in this scenario. It says, “A tail-log backup of the source database will be taken.” Sounds innocuous if you don’t know what that implies. In past versions, restoring a copy of a database would never affect the original – or not by default, anyway – so I cringe as I imagine people merrily clicking past this warning.

The script you get with these settings is indeed composed with a tail-log backup of the source database:

That will, as indicated, take a tail-log backup – and thereby take the original database down. Nice if you wanted that, but a mean surprise if you didn’t.

If you act on the warning and click the Options tab, and then uncheck the offending setting, you do get the expected behavior (the original database is unaffected because the tail-log backup is not included at the top of the script):

So, be careful out there!

March Update to Rules-Driven Maintenance

merrillaldrich — Tue, 12 Mar 2013 02:56:00 GMT

This month I have a minor update to the Rules-Driven Maintenance code I originally posted back in August 2012. This update has just two enhancements, but they are nice ones, I think:

Much improved handling for instances that use database snapshots.
Ability to do intra-day differential backups purely by setting maintenance times and limits in the policy table.

The code posted here is cumulative, and replaces entirely the original code, but please refer back to that original blog post for description, instructions and details. This update can be installed right over an existing deployment, or be installed all by itself as a complete solution but – as always – please test and be cautious.

Enjoy!

Update to Rules-Driven Maintenance

merrillaldrich — Wed, 16 Jan 2013 18:34:00 GMT

Back in August I posted a first version of a rules-driven solution for backups, index and statistics maintenance and integrity checks. The system in general has been working well, and has saved my team a huge amount of time and effort. We are coming to the anniversary of its use in production soon.

Today I offer an update that contains a few enhancements, performance improvements and a bug fix.

To recap, this is a system or framework to manage many small databases across many instances on many servers in a policy-based/automated way. The details about how to deploy and how to use the system are in the original post, but at a high level:

First deploy the code across a collection of servers using the included PowerShell script DeployMaintenance.ps1. That will create an administrative database on every instance (or use one that you provide) and populate it with the required tables, procedures, etc. The deployment script will also make a handful of SQL Agent jobs that handle all the maintenance on all the databases.
Visit the instances. If there is maintenance already in place, disable or remove it so as not to perform this work twice. If necessary, change or override the default preferences for this solution in the admin database for each type of maintenance. This includes the time of maintenance windows, whether to use features like differential backups or Litespeed, whether there are databases that should be treated as exceptions, with different settings, etc.
Enable the new policy jobs in SQL Agent.
Monitor to make sure things are running smoothly.

The theory behind this project is to save work configuring servers (efficiency), to ensure that the maintenance is in fact deployed and working for ALL databases in the environment (quality), to ensure that it works in a truly consistent way everywhere (consistency), and to make it so that the servers don’t have to be touched by a DBA when changes happen, such as adding or dropping databases (resiliency).

Changes

The updates in this 1.1 version include

Better reporting of errors. The jobs now write log files to the default error log location for SQL Server.
Better recovery from errors. The jobs in some cases continue on error – for example, one failed backup will not stop the backup job and leave other databases without backups.
Performance enhancements for examining backup history and system counters (mainly % log used).
One bug fix for the log backup logic on a server that uses both log shipping and Litespeed.

Disclaimers

The code for the system is posted here, and you are welcome to download it and try it out. Please note that this is not a commercial solution, and that while you may test or deploy this, you do so at your sole risk. Edits to the code almost certainly will be required for your environment. Please read, understand and test the code thoroughly before you even imagine using this in production. I do not want to cause you or your employer any pain. License for using this solution is GPL 3.0: http://opensource.org/licenses/GPL-3.0. You may use and modify this code, but not sell it.

This has been run on SQL Server versions from 2005 to 2008 R2. It probably works on 2012, but I have not tested it thoroughly.

I run only case-insensitive collation; if you are on a case-sensitive system, then you have my sympathy, and you might have to edit a few things to make this work.

There is coverage for Quest Litespeed backups in the solution; other third-party backup products could be incorporated easily, but have not been. It should be possible to follow the pattern used to handle Litespeed and extend it to any other third-party stored procedures.

If you haven’t done index maintenance for a while, and you flip this solution on you might blow up your transaction log. Be sure that you have plenty of log space and plenty of log backup space, and watch the process the first few runs. This is especially true if the combination of Standard Edition of SQL Server and the preference for Online index maintenance applies in your environment, because the maintenance process will perform index maintenance using reorganize (defrag), which produces a lot of log.

Lastly, this is a solution for the type of environment that has many small databases, not the type with a few big, performance intensive databases. Databases that are large and require detailed attention may not be suitable for a one-size-fits-all solution like this.

Quick Tip - Speed a Slow Restore from the Transaction Log

KKline — Wed, 14 Nov 2012 15:59:00 GMT

Here's a quick tip for you:

During some restore operations on Microsoft SQL Server, the transaction log redo step might be taking an unusually long time. Depending somewhat on the version and edition of SQL Server you've installed, you may be able to increase performance by tinkering with the readahead performance for the redo operations. To do this, you should use the MAXTRANSFERSIZE parameter of the RESTORE statement. For example, if you set MAXTRANSFERSIZE=1048576, it'll use 1MB buffers.

If you change the MAXTRANSFERSIZE, keep an eye on the PerfMon objects for Buffer Manager and Readahead IO. You may also wish to keep an eye on LOGBUFFER wait stats.

I'd love to hear your feedback. Have you tried this technique? Did it work as advertised? Did it require some changes to work on a specific version or edition?

Many thanks,

-Kev

-Follow me on Twitter!

Backup File Naming Convention

Andrew Kelly — Fri, 05 Oct 2012 19:41:00 GMT

I have been asked this many times before and again just recently so I figured why not blog about it. None of this information outlined here is rocket science or even new but it is an area that I don’t think people put enough thought into before implementing. Sure everyone choses some format but it often doesn’t go far enough in my opinion to get the most bang for the buck. This is the format I prefer to use:

ServerName_InstanceName_BackupType_DBName_DateTimeStamp_nn.xxx

ServerName_InstanceName = Pretty self explanatory but lets look at it. Let’s say that the machine name is M432 and the instance is Dev2008. That would normally be W432\Dev2008 however I don’t like special characters so I change it to W432_Dev2008. If it was a default instance it would be W432_W432. Some people (including myself) prefer to leave off the Server Name if it is a default instance but that is up to you. Since the default instance is always the name of the server it’s still pretty clear where it came from.

BackupType = FULL, DIFF or LOG. Nothing more is needed here.

DBName = Full name of the database. One note here is that if the name has spaces I like to replace the space with some other valid character and some people prefer to remove the space altogether. Spaces in an object name is a whole debate in itself and I wont go there now .

DateTimeStamp = yyyymmddhhmmss. This allows me to know exactly when the backup started just by looking at the name and makes it unique as well. I don’t know any one who takes two backups of the same db in less than a second so this convention works to avoid file name conflicts.
_nn = The individual number associated with backing up to multiple files for a single backup operation. Typically backing up to multiple files for a FULL or DIFF backup can be more effecient with larger dbs so appending a number from 01 to nn ensures uniqueness as the rest of the name will be the same. If it is a single file then you can simply use 01 or omit that part altogether.

.xxx = I also like to use the extension to identify the tool used to create the backup file. For native SQL Server backups I use .bak regardless of the type of backup (Log, Diff or Full). For backups done using a 3rd party utility such as the one from Red-Gate I would use .sqb and so on.

This naming convention allows me to do several things. First it ensures each backup file will have a unique name. Second it allows me with a quick glance to see where the backup originated, what type of backup it is, which database it is for, when the backup started and which tool was used to create the backup. So again there is nothing particularly new to this approach but I often see the naming falling short of this and generally only having the DB name and timestamp. Why not take the extra few steps to ensure you get the most out of your naming that you can. The code to generate the whole file name is pretty simple and can be done dynamically so why not go this route? OK there are always exceptions so let’s not start a debate war .

Have fun,

Andy

Real tortoises keep it slow and steady. How about the backups?

Maria Zakourdaev — Mon, 10 Sep 2012 10:54:00 GMT

… Four tortoises were playing in the backyard when they decided they needed hibiscus flower snacks. They pooled their money and sent the smallest tortoise out to fetch the snacks.
Two days passed and there was no sign of the tortoise.
"You know, she is taking a lot of time", said one of the tortoises.
A little voice from just out side the fence said, "If you are going to talk that way about me I won't go."

Is it too much to request from the quite expensive 3rd party backup tool to be a way faster than the SQL server native backup?

Or at least save a respectable amount of storage by producing a really smaller backup files? By saying “really smaller”, I mean at least getting a file in half size.

After Googling the internet in an attempt to understand what other “sql people” are using for database backups, I see that most people are using one of three tools which are the main players in SQL backup area:

LiteSpeed by Quest
SQL Backup by Red Gate
SQL Safe by Idera

The feedbacks about those tools are truly emotional and happy. However, while reading the forums and blogs I have wondered, is it possible that many are accustomed to using the above tools since SQL 2000 and 2005.
This can easily be understood due to the fact that a 300GB database backup for instance, using regular a SQL 2005 backup statement would have run for about 3 hours and have produced ~150GB file (depending on the content, of course).
Then you take a 3rd party tool which performs the same backup in 30 minutes resulting in a 30GB file leaving you speechless, you run to management persuading them to buy it due to the fact that it is definitely worth the price.
In addition to the increased speed and disk space savings you would also get backup file encryption and virtual restore - features that are still missing from the SQL server.

But in case you, as well as me, don’t need these additional features and only want a tool that performs a full backup MUCH faster AND produces a far smaller backup file (like the gain you observed back in SQL 2005 days) you will be quite disappointed. SQL Server backup compression feature has totally changed the market picture.

Medium size database.

Take a look at the table below, check out how my SQL server 2008 R2 compares to other tools when backing up a 300GB database.

It appears that when talking about the backup speed, SQL 2008 R2 compresses and performs backup in similar overall times as all three other tools. 3rd party tools maximum compression level takes twice longer.
Backup file gain is not that impressive, except the highest compression levels but the price that you pay is very high cpu load and much longer time. Only SQL Safe by Idera was quite fast with it’s maximum compression level but most of the run time have used 95% cpu on the server.
Note that I have used two types of destination storage, SATA 11 disks and FC 53 disks and, obviously, on faster storage have got my backup ready in half time.

Looking at the above results, should we spend money, bother with another layer of complexity and software middle-man for the medium sized databases? I’m definitely not going to do so.

Very large database

As a next phase of this benchmark, I have moved to a 6 terabyte database which was actually my main backup target.

Note, how multiple files usage enables the SQL Server backup operation to use parallel I/O and remarkably increases it’s speed, especially when the backup device is heavily striped. SQL Server supports a maximum of 64 backup devices for a single backup operation but the most speed is gained when using one file per CPU, in the case above 8 files for a 2 Quad CPU server. The impact of additional files is minimal.
However, SQLsafe doesn’t show any speed improvement between 4 files and 8 files.

Of course, with such huge databases every half percent of the compression transforms into the noticeable numbers. Saving almost 470GB of space may turn the backup tool into quite valuable purchase. Still, the backup speed and high CPU are the variables that should be taken into the consideration.

As for us, the backup speed is more critical than the storage and we cannot allow a production server to sustain 95% cpu for such a long time.
Bottomline, 3rd party backup tool developers, we are waiting for some breakthrough release.

There are a few unanswered questions, like the restore speed comparison between different tools and the impact of multiple backup files on restore operation. Stay tuned for the next benchmarks.

Benchmark server:

SQL Server 2008 R2 sp1
2 Quad CPU
Database location: NetApp FC 15K Aggregate 53 discs
Backup destination volumes: two physical NetApps FC 15K Aggregate 53 discs, 4 files on each volume.

Backup statements:

No matter how good that UI is, we need to run the backup tasks from inside of SQL Server Agent to make sure they are covered by our monitoring systems. I have used extended stored procedures (command line execution also is an option, I haven’t noticed any impact on the backup performance).

SQL backup

LiteSpeed

SQL Backup

SQL safe

backup database <DBNAME> to

disk= '\\<networkpath>\par1.bak' ,

disk= '\\<networkpath>\par2.bak',

disk= '\\<networkpath>\par3.bak'

with format, compression

EXECUTE master.dbo.xp_backup_database

@database = N'<DBName>',

@backupname= N'<DBName> full backup',

@desc = N'Test',

@compressionlevel=8,

@filename= N'\\<networkpath>\par1.bak',

@filename= N'\\<networkpath>\par2.bak',

@filename= N'\\<networkpath>\par3.bak',
@init = 1

EXECUTE master.dbo.sqlbackup
'-SQL "BACKUP DATABASE <DBNAME>

TO DISK= ''\\<networkpath>\par1.sqb'',

DISK= ''\\<networkpath>\par2.sqb'',

DISK= ''\\<networkpath>\par3.sqb''

WITH

DISKRETRYINTERVAL = 30,
DISKRETRYCOUNT = 10,
COMPRESSION = 4,
INIT"'

EXECUTE master.dbo.xp_ss_backup

@database = 'UCMSDB',

@filename = '\\<networkpath>\par1.bak',

@backuptype = 'Full',

@compressionlevel = 4,

@backupfile = '\\<networkpath>\par2.bak',

@backupfile = '\\<networkpath>\par3.bak'

If you still insist on using 3rd party tools for the backups in your production environment with maximum compression level, you will definitely need to consider limiting cpu usage which will increase the backup operation time even more:

RedGate : use THREADPRIORITY option ( values 0 – 6 )
LiteSpeed : use @throttle ( percentage, like 70%)
SQL safe : the only thing I have found was @Threads option.

Yours,

Maria

High-Availability White Papers and Resources for SQL Server

KKline — Thu, 26 Jul 2012 15:00:00 GMT

In foreground, attendee makes dreaded "shoot myself" hand sign to the speaker.

I was just telling the good people of Charlotte about how they (and how YOU) need to read all things by Paul Randal (blog | twitter), except for all of his cheesy romance novels like Caress and Conquer written under the nom de plum of Connie Mason.

There's lots more good stuff from Paul, just not romantic.

This is a 'so-last-version' whitepaper describing five common high-availability and disaster-recovery architectures deployed by customers, along with a case study of each. Although the white paper is specific to SQL Server 2008 R2 and isn't updated for AlwaysOn features, it's still really, really good. It covers:

Failover Clustering for High Availability with Database Mirroring for Disaster Recovery
Database Mirroring for High Availability and Disaster Recovery
Geo-Clustering for High Availability and Disaster Recovery
Failover Clustering for High Availability Combined with SAN-Based Replication for Disaster Recovery
Peer-to-Peer Replication for High Availability and Disaster Recovery

You can get it from this link. Not everything is transferable to new AlwaysOn technologies, but then again AlwaysOn is an Enterprise Edition feature. So the database mirroring recommendation can be upsized, in many if not all cases, to SQL Server 2012, while the SAN and peer-to-peer recommendations continue to hold fast.

In addition, I encourage you to get up to speed on AlwaysOn. There are two great AlwaysOn FAQs that I recommend. The first is Microsoft’s official AlwaysOn FAQ at http://msdn.microsoft.com/en-us/sqlserver/gg508768.aspx. The second comes from my buddy and high-availability expert Allan Hirt (blog | twitter) at http://www.sqlha.com/2012/04/13/allans-alwayson-availability-groups-faq/.

To get started with AlwaysOn, check out http://msdn.microsoft.com/en-us/library/cc645581.aspx.

Enjoy,

-Kev

Backup and the evil RETAINDAYS option

TiborKaraszi — Sun, 08 Jul 2012 12:25:00 GMT

"So what bad has this option done?", you probably as yourself. Well, not much, but I find it evil because it confuses people, especially those new to SQL Server. I have many times seen people specifying something like 3, and expect SQL Server to keep the three most recent backups in the backup file and overwrite everything which is older than that. Well, that is not what the option does.

But before we go into details, let's look at an example backup command which is using this option:

BACKUP DATABASE sqlmaint TO DISK = 'R:\sqlmaint.bak' WITH RETAINDAYS = 3

The RETAINDAYS is also exposed in the backup dialog in SSMS: "Backup set will expire: After x days".

It is also exposed in Maintenance Plans, the backup task. The option is named "Backup set will expire: After x days". It is only enabled if you select the "Back up databases across one or more files" option, which is not the default option. This makes sense.
The default option is "Create a backup files for every database", which means that every time a backup is performed, a new file is created consisting of the database name, date and time. Since we will see that this option is only relevant when we do append, it makes sense in the RETAINDAYS not being enabled for this choice.

So what does this option do? All it does is make SQL Server return an error message of you try to do a backup using the INIT option (which means overwrite) before the date and time has occurred. In other words, it tries to help you in not overwriting a backup file, using the INIT option, before it is time. You can still overwrite earlier by either using the stronger FORMAT option instead of INIT; or by simply deleting the backup file. Btw, the RETAINDAYS parameter has a cousin named EXPIREDATE, which does the same thing but you specify a datetime value instead of number of days.

Backup generations
So, we have seen that RETAINDAYS do not in any way provide any automatic backup generation handling. There is no such functionality built-in in the BACKUP command. This means that when you are looking for functionality such as "keep backup files three days, and remove older than that", you need to look outside the BACKUP command. I.e., some script or tool of some sort.

I have an example (without the "delete old files" part) here, which I mostly created as a starting point for those who want to roll their own and want to have some example to start with. Many of you are probably using Maintenance plans (the "Create a backup files for every database" option in the backup task, along with Maintenance Cleanup task). Another popular script/tool for this is Ola Hallengren's Maintenance Solution, which you find at http://ola.hallengren.com/.