In a followup to Aaron Bertrand's recent blog post on Disaster Recovery, I'd like to mention something I always mention when I teach SQL Server classes.
One of the best sources of training I received for being a DBA was, in fact, the training I received to become a pilot. The two things I took from pilot training are the use of checklists, which I blogged about here, and practicing Emergency Procedures.
When you learn to fly you spend the majority of your time in the air learning how to handle the things that can go wrong. Anyone can fly an airplane when things go smoothly, but the real pilots are the ones who keep their cool and handle things when things go wrong. They are able to do this because they practice.
Likewise, a good DBA not only has a solid backup strategy, but has actually TESTED those backups to ensure they're usable should the need arise. High Availability failover strategies are not only configured, but tested, both at the database and server level and at the application level, to ensure that the applications continue to support the business when a failover occurs.
It would be useful to make a list of all the ways in which things can go wrong, and develop a checklist on procedures to solve those problems. Use that checklist to practice recovering from those problems so that when they do occur you can calmly and professionally handle the problem.
The important thing to remember is to practice your solutions.
Allen