Monday, October 19, 2009

SQL Server database recovery model

Hi,

Today we will discuss about various recovery models of SQL Server. A recovery plan is as important as water for a person who is searching for land in a desert. A recovery plan help to restore database whenever there is a database failure.

Each database on database server can be setup differently and have the ability to change the recovery model as needed.

Let's discuss the three plans in detail:

Simple

The simple recovery model gives a simple backup that can be used to replace our entire database in the event of a failure or if we have a need to restore your database to another server. With this recovery model we have the ability to do complete backups (an entire copy) or differential backups (any changes since the last complete backup). With this recovery model we are exposed to any failures since the last backup completed.

Why we may choose this recovery model:



  • Your data is not critical and can easily be recreated



  • The database is only used for test or development



  • Data is static and does not change



  • Losing any or all transactions since the last backup is not a problem



  • Data is derived and can easily be recreated




  • Type of backups:


  • Complete backups



  • Differential backups



  • File and/or Filegroup backups



  • Partial backups



  • Copy-Only backups




  • Bulk_Logged

    With Bulk_Logged recovery model, bulk operations such as BULK INSERT, CREATE INDEX, SELECT INTO, etc... that are not fully logged in the transaction log and therefore do not take as much space in the transaction log, are logged. The advantage of using this recovery model is that our transaction logs will not get that large if you are doing bulk operations and we have the ability to do point in time recovery as long as our last transaction log backup does not include a bulk operation as mentioned above. If no bulk operations are run this recovery model works the same as the Full recovery model. One thing to note is that if we use this recovery model we also need to issue transaction log backups otherwise our database transaction log will continue to grow.

    Here are some reasons why we may choose this recovery model:



  • Data is critical, but you do not want to log large bulk operations



  • Bulk operations are done at different times versus normal processing.



  • You still want to be able to recover to a point in time



  • Type of backups you can run:


  • Complete backups



  • Differential backups



  • File and/or Filegroup backups



  • Partial backups



  • Copy-Only backups



  • Transaction log backups




  • Full

    The full recovery model is the most complete recovery model and allows to recover all data to any point in time as long as all backup files are useable. With this model all operations are fully logged which means that can recover our database to any point. In addition, if the database is set to the full recovery model we need to also issue transaction log backups otherwise our database transaction log will continue to grow forever.

    Here are some reasons why you may choose this recovery model:



  • Data is critical and data can not be lost.



  • You always need the ability to do a point-in-time recovery.



  • You are using database mirroring



  • Type of backups you can run:


  • Complete backups



  • Differential backups



  • File and/or Filegroup backups



  • Partial backups



  • Copy-Only backups



  • Transaction log backups



  • How to update / select Recovery Models

    The recovery model can be changed as needed, so if your database is in the Full recovery model and you want to issue some bulk operations that you want to minimally log you can change the recovery model to Bulk_Logged complete your operations and then change your database model again. One thing to note is that since there will be a bulk operation in your transaction log, in backup you can not do a point in time recovery using this transaction log backup file that contains this bulk operation, but any subsequent transaction log backup can be used to do a point in time recovery.

    Also, if your database is in the Simple recovery model and you change to the Full recovery model you will want to issue a full backup immediately, so you can then begin to also do transaction log backups. Until you issue a full backup you will not be able to take transaction log backups.

    To change the recovery model you can use either SQL Server Management Studio or T-SQL as follows:

    Management Studio

    Right click on the database name, select Properties, select the Options tab and select recovery model from the drop-down list and select OK to save.



    T-SQL

    -- set to Full recovery
    ALTER DATABASE AdventureWorks SET RECOVERY FULL
    GO
    -- set to Bulk Logged recovery
    ALTER DATABASE AdventureWorks SET RECOVERY BULK_LOGGED
    GO
    -- set to Simple recovery
    ALTER DATABASE AdventureWorks SET RECOVERY SIMPLE
    GO

    What is TempDB database?

    In SQL Server 2005, TempDB plays a very important role and some of the best practices have changed and so has the necessity to follow these best practices on a more wide scale basis. In many cases TempDB has been left to default configurations in many of our SQL Server installations. Unfortunately, these configurations are not necessarily ideal in many environments.

    Let's see how TempDB can be optimized to improve the overall SQL Server performance.

    Responsibilities of TempDB

  • Global (##temp) or local (#temp) temporary tables, temporary table indexes, temporary stored procedures, table variables, tables returned in table-valued functions or cursors.

  • Database Engine objects to complete a query such as work tables to store intermediate results for spools or sorting from particular GROUP BY, ORDER BY, or UNION queries.

  • Row versioning values for online index processes, Multiple Active Result Sets (MARS) sessions, AFTER triggers and index operations (SORT_IN_TEMPDB).

  • DBCC CHECKDB work tables.

  • Large object (varchar(max), nvarchar(max), varbinary(max) text, ntext, image, xml) data type variables and parameters.


  • Best practices for TempDB are:

  • Do not change collation from the SQL Server instance collation.

  • Do not change the database owner from sa.

  • Do not drop the TempDB database.

  • Do not drop the guest user from the database.

  • Do not change the recovery model from SIMPLE.

  • Ensure the disk drives TempDB resides on have RAID protection i.e. 1, 1 + 0 or 5 in order to prevent a single disk failure from shutting down SQL Server. Keep in mind that if TempDB is not available then SQL Server cannot operate.

  • If SQL Server system databases are installed on the system partition, at a minimum move the TempDB database from the system partition to another set of disks.

  • Size the TempDB database appropriately. For example, if you use the SORT_IN_TEMPDB option when you rebuild indexes, be sure to have sufficient free space in TempDB to store sorting operations. In addition, if you are running into insufficient space errors in TempDB, be sure to determine the culprit and either expand TempDB or re-code the offending process.


  • Hope it will help develop some love for TempDB also.

    Saturday, October 17, 2009

    Derived Tables

    Sometimes querying data is not that simple and there may be the need to create temporary tables or views to predefine how the data should look prior to its final output. Unfortunately there are problems with both of these approaches if you are trying to query data on the fly.

    Temporary table approach
    With the temporary tables approach you need to have multiple steps in your process, first to create the temporary table, then to populate the temporary table, then to select data from the temporary table and lastly cleanup of the temporary table.

    View approach
    With the view approach you need to predefine how this data will look, create the view and then use the view in your query. Granted if this is something that you would be doing over and over again this might make sense to just create a view, but let's look at a totally different approach.

    SQL Server provides allows to create derived tables on the fly and then use these derived tables within your query. This is similar to creating a temporary table and then using the temporary table in your query.

    Let's take a look at an example where we query Northwind database to find out how many customers fall into various categories based on sales. The categories that we have predefined are as follows:

    Total Sales between 0 and 5,000 = Micro
    Total Sales between 5,001 and 10,000 = Small
    Total Sales between 10,001 and 15,000 = Medium
    Total Sales between 15,001 and 20,000 = Large
    Total Sales > 20,000 = Very Large


    There are several ways to get this data but we will use derived tables approach.

    The first step is to find out the total sales by each customer, which can be done with the following statement.

    SELECT o.CustomerID,
    SUM(UnitPrice * Quantity) AS TotalSales
    FROM [Order Details] AS od
    INNER JOIN Orders AS o
    ON od.OrderID = o.OrderID
    GROUP BY o.CustomerID

    This is a partial list of the output:

    CustomerID TotalSales
    -------------------------------
    ALFKI 4596.2000
    ANATR 1402.9500
    ANTON 7515.3500
    WOLZA 3531.9500

    Now classify the TotalSales value into the OrderGroups that was specified above:

    SELECT o.CustomerID,
    SUM(UnitPrice * Quantity) AS TotalSales,
    CASE
    WHEN SUM(UnitPrice * Quantity)
    BETWEEN 0 AND 5000 THEN 'Micro'
    WHEN SUM(UnitPrice * Quantity)
    BETWEEN 5001 AND 10000 THEN 'Small'
    WHEN SUM(UnitPrice * Quantity)
    BETWEEN 10001 AND 15000 THEN 'Medium'
    WHEN SUM(UnitPrice * Quantity)
    BETWEEN 15001 AND 20000 THEN 'Large'
    WHEN SUM(UnitPrice * Quantity)
    > 20000 THEN 'Very Large'
    END AS OrderGroup
    FROM [Order Details] AS od
    INNER JOIN Orders AS o
    ON od.OrderID = o.OrderID
    GROUP BY o.CustomerID

    This is a partial list of the output:

    CustomerID TotalSales OrderGroup

    -------------------------------------------
    ALFKI 4596.2000 Micro
    ANATR 1402.9500 Micro
    ANTON 7515.3500 Small
    WOLZA 3531.9500 Micro

    There can be many customers who fit into each of these groups and this is where the derived table comes into play. What we are doing here is using the same query from the step above, but using it as derived table OG. Then we are selecting data from this derived table for our final output just like we would with any other query. All of the columns that are created in the derived table are now available in our final query.


    SELECT OG.OrderGroup,
    COUNT(OG.OrderGroup) AS OrderGroupCount
    FROM (SELECT o.CustomerID,
    SUM(UnitPrice * Quantity) AS TotalSales,
    CASE
    WHEN SUM(UnitPrice * Quantity)
    BETWEEN 0 AND 5000 THEN 'Micro'
    WHEN SUM(UnitPrice * Quantity)
    BETWEEN 5001 AND 10000 THEN 'Small'
    WHEN SUM(UnitPrice * Quantity)
    BETWEEN 10001 AND 15000 THEN 'Medium'
    WHEN SUM(UnitPrice * Quantity)
    BETWEEN 15001 AND 20000 THEN 'Large'
    WHEN SUM(UnitPrice * Quantity)
    > 20000 THEN 'Very Large'
    END AS OrderGroup
    FROM [Order Details] AS od
    INNER JOIN Orders AS o
    ON od.OrderID = o.OrderID
    GROUP BY o.CustomerID) AS OG
    GROUP BY OG.OrderGroup

    This is the complete list of the output from the above query.

    OrderGroup OrderGroupCount
    -----------------------------------
    Large 10
    Medium 11
    Micro 33
    Small 15
    Very Large 20

    Dropping multiple objects using a single DROP statement

    Today, let's see how we can drop multiple objects with single statement.

    Almost every SQL Server object that is created may need to be dropped at some time, especially when you are developing. You create a bunch of temporary objects which you do not want to keep in the database for long. Most SQL Server users drop one object at a time using either SSMS or a drop statement. In many scenarios we may need to drop several objects of the same type. Is there a way to drop several objects through less lines of code?

    With T-SQL we can drop multiple objects of the same type through a single drop statement. Almost any object that can be dropped in a single drop statement can also be dropped simultaneously with other objects of the same type through one drop statement.

    Some of these include objects like databases, tables, functions, stored procedures, rules, synonyms etc.

    Let's look at an example.

    First we create a few stored procedures, so we can test single and multiple drops.

    Script # 1: Create 4 stored procedures

    USE AdventureWorks
    GO
    CREATE PROCEDURE USP1
    AS
    BEGIN
    SELECT TOP 10 * FROM Person.Address
    END
    GO
    CREATE PROCEDURE USP2
    AS
    BEGIN
    SELECT TOP 10 * FROM Person.Address
    END
    GO
    CREATE PROCEDURE USP3
    AS
    BEGIN
    SELECT TOP 10 * FROM Person.Address
    END
    GO
    CREATE PROCEDURE USP4
    AS
    BEGIN
    SELECT TOP 10 * FROM Person.Address
    END
    GO

    Now we have 4 stored procedures to work with.

    Let's drop the first three using a single drop statement as shown below.

    Script # 2: Drop USP1, USP2, USP3 through three drop statements

    USE AdventureWorks
    GO
    DROP PROCEDURE USP1
    DROP PROCEDURE USP2
    DROP PROCEDURE USP3
    GO

    Let's create USP1, USP2 and USP3 again.

    USE AdventureWorks
    GO
    CREATE PROCEDURE USP1
    AS
    BEGIN
    SELECT TOP 10 * FROM Person.Address
    END
    GO
    CREATE PROCEDURE USP2
    AS
    BEGIN
    SELECT TOP 10 * FROM Person.Address
    END
    GO
    CREATE PROCEDURE USP3
    AS
    BEGIN
    SELECT TOP 10 * FROM Person.Address
    END
    GO

    The following script will drop multiple stored procedures through one drop statement. We can see that we just need to put the list of objects to drop and separate them with a comma. as shown below. The rest of the syntax is the same.

    Script # 3: Drop USP1, USP2, USP3 through single drop statement
    USE AdventureWorks
    GO
    DROP PROCEDURE USP1,USP2,USP3
    GO

    Through Script # 3 USP1, USP2 and USP3 have been dropped in a single drop statement.

    Following are some of the pros and cons of multiple object drops:

    Benefits


  • The multiple objects drop approach is applicable to all versions of SQL Server.



  • If some objects in the list do not exist or can not be dropped due to privileges or they do not exist, the remaining objects will be successfully dropped without any negative impact.



  • Although no query plan is generated for drop statements, you can see the dropping of multiple objects approach consumes less bytes while requesting data over the network. This can be verified from network statistics while client statistics are enabled in SQL Server Management Studio (SSMS).



  • Through less lines of code you can get more done.



  • Short Comings


  • It is not possible to apply pre-existence check for the objects you want to drop, such as IF EXISTS



  • It should be obvious, but good to mention that you can not drop objects of different types together in a single statement. For example you can not drop tables and stored procedures at the same time.



  • Hope this way we can reduce our statements and reduce network traffic.

    Friday, October 16, 2009

    Tables variables & how its different from Temp Table

    Hello Friends,


    I am sure most of you would have heard of table variables, but not sure how to use them in a stored procedure. The question would be What purpose do they serve and why not just use temporary tables instead?

    let's discuss this today.

    If you already know how to create and use a temporary table then you're going to have no problem understanding how to use a table variable. The usage is just about identical.

    Temporary Tables

    Temporary tables are created in tempdb. The name "temporary" is slightly misleading, for even though the tables are instantiated in tempdb, they are backed by physical disk and are even logged into the transaction log. They act like regular tables in that you can query their data via SELECT queries and modify their data via UPDATE, INSERT, and DELETE statements. If created inside a stored procedure they are destroyed upon completion of the stored procedure. Furthermore, the scope of any particular temporary table is the session in which it is created; meaning it is only visible to the current user. Multiple users could create a temp table named #TableX and any queries run simultaneously would not affect one another - they would remain autonomous transactions and the tables would remain autonomous objects. You may notice that my sample temporary table name started with a "#" sign. This is the identifier for SQL Server that it is dealing with a temporary table.

    The syntax for creating a temporary table is identical to creating a physical table in Microsoft SQL Server with the exception of the aforementioned pound sign (#):

    CREATE TABLE dbo.#Cars
    (
    Car_id int NOT NULL,
    ColorCode varchar(10),
    ModelName varchar(20),
    Code int,
    DateEntered datetime
    )

    Temporary tables act like physical tables in many ways. You can create indexes and statistics on temporary tables. You can also apply Data Definition Language (DDL) statements against temporary tables to add constraints, defaults, and referential integrity such as primary and foreign keys. You can also add and drop columns from temporary tables. For example, if I wanted to add a default value to the DateEntered column and create a primary key using the Car_id field I would use the following syntax:

    ALTER TABLE dbo.#Cars
    ADD
    CONSTRAINT [DF_DateEntered] DEFAULT (GETDATE()) FOR [DateEntered],
    PRIMARY KEY CLUSTERED
    ( [Car_id] ) ON [PRIMARY]
    GO

    Table Variables

    The syntax for creating table variables is quite similar to creating either regular or temporary tables. The only differences involve a naming convention unique to variables in general, and the need to declare the table variable as you would any other local variable in Transact SQL:

    DECLARE @Cars table (
    Car_id int NOT NULL,
    ColorCode varchar(10),
    ModelName varchar(20),
    Code int,
    DateEntered datetime )

    As you can see the syntax bridges local variable declaration (DECLARE @variable_name variable_data_type) and table creation (column_name, data_type, nullability). As with any other local variable in T-SQL, the table variable must be prefixed with an "@" sign. Unlike temporary or regular table objects, table variables have certain clear limitations.



  • Table variables can not have Non-Clustered Indexes


  • You can not create constraints in table variables


  • You can not create default values on table variable columns


  • Statistics can not be created against table variables




  • Similarities with temporary tables include:


  • Instantiated in tempdb


  • Clustered indexes can be created on table variables and temporary tables


  • Both are logged in the transaction log


  • Just as with temp and regular tables, users can perform all Data Modification Language (DML) queries against a table variable: SELECT, INSERT, UPDATE, and DELETE.





  • Temporary tables are usually preferred over table variables for a few important reasons: they behave more like physical tables in respect to indexing and statistics creation and lifespan. An interesting limitation of table variables comes into play when executing code that involves a table variable. The following two blocks of code both create a table called #Cars and @Cars. A row is then inserted into the table and the table is finally queried for its values.

    --Temp Table:
    CREATE TABLE dbo.#Cars
    (
    Car_id int NOT NULL,
    ColorCode varchar(10),
    ModelName varchar(20),
    Code int ,
    DateEntered datetime
    )

    INSERT INTO dbo.#Cars (Car_id, ColorCode, ModelName, Code, DateEntered)
    VALUES (1,'BlueGreen', 'Austen', 200801, GETDATE())

    SELECT Car_id, ColorCode, ModelName, Code, DateEntered FROM dbo.#Cars

    DROP TABLE dbo.[#Cars]

    This returns the following results:





    --Table Variable:

    DECLARE @Cars TABLE
    ( Car_id int NOT NULL,
    ColorCode varchar(10),
    ModelName varchar(20),
    Code int ,
    DateEntered datetime )

    INSERT INTO @Cars (Car_id, ColorCode, ModelName, Code, DateEntered)
    VALUES (1,'BlueGreen', 'Austen', 200801, GETDATE())

    SELECT Car_id, ColorCode, ModelName, Code, DateEntered FROM @Cars
     
    The results differ, depending upon how you run the code. If you run the entire block of code the following results are returned:
     
     

     

    However, you receive an error if you don't execute all the code simultaneously:

    Msg 1087, Level 15, State 2, Line 1
    Must declare the table variable "@Cars"

    What is the reason for this behavior? It is quite simple. A table variable's lifespan is only for the duration of the transaction that it runs in. If we execute the DECLARE statement first, then attempt to insert records into the @Cars table variable we receive the error because the table variable has passed out of existence. The results are the same if we declare and insert records into @Cars in one transaction and then attempt to query the table. If you notice, we need to execute a DROP TABLE statement against #Cars. This is because the table persists until the session ends or until the table is dropped.

    Table variables serve a very useful purpose in returning results from table value functions. Take for example the following code for creating a user-defined function that returns values from the Customers table in the Northwind database for any customers in a given PostalCode:

    CREATE FUNCTION dbo.usp_customersbyPostalCode ( @PostalCode VARCHAR(15) )
    RETURNS
    @CustomerHitsTab TABLE (
    [CustomerID] [nchar] (5),
    [ContactName] [nvarchar] (30),
    [Phone] [nvarchar] (24),
    [Fax] [nvarchar] (24)
    )
    AS
    BEGIN
    DECLARE @HitCount INT

    INSERT INTO @CustomerHitsTab
    SELECT [CustomerID],
    [ContactName],
    [Phone],
    [Fax]
    FROM [Northwind].[dbo].[Customers]
    WHERE PostalCode = @PostalCode

    SELECT @HitCount = COUNT(*) FROM @CustomerHitsTab

    IF @HitCount = 0
    --No Records Match Criteria
    INSERT INTO @CustomerHitsTab (
    [CustomerID],
    [ContactName],
    [Phone],
    [Fax] )
    VALUES ('','No Companies In Area','','')

    RETURN
    END
    GO


    The @CustomerHitsTab table variable is created for the purpose of collecting and returning results of a function to the end user calling the dbo.usp_customersbyPostalCode function.


    SELECT * FROM dbo.usp_customersbyPostalCode('1010')



    SELECT * FROM dbo.usp_customersbyPostalCode('05033')





    An unofficial rule-of-thumb for usage is to use table variables for returning results from user-defined functions that return table values and to use temporary tables for storage and manipulation of temporary data; particularly when dealing with large amounts of data. However, when lesser row counts are involved, and when indexing is not a factor, both table variables and temporary tables perform comparably. It then comes down to preference of the individual responsible for the coding process.


    Happy SQL Coding