MCITP

MCITP

Friday, April 11, 2014

Log shipping breaks what to do now


Log shipping breaks what to do now :
----------------------------------------------------------

1.       Either we can reconfigure log shipping using full and log backup.

2.       Or if lucky we can take one differential backup and keep log shipping active but there is a catch:
We must make sure there is no full backup performed on primary database after log shipping breaks and before taking differential backup.

Reason :

A differential backup can be restored on database
a.       After last full back up has been restored on secondary server  with norecovery or
b.      Any previous(old) full backup together with all subsequent log backup and atleast one log backup after last full backup and before final diff backup taken has been restored before another full backup

Consider the following scenario for restoring differential backup (DF3) take at today (T) at 7 PM
DB1                                                                                                                                        DB2

Backup type (Base LSN)
Time
Restored database (Base LSN)
Full backup  (..031) FB1
T-10
(..031)
Log backup  (..031)  LB1
T-9
(..031)
Log backup  (..031)  LB2
T-8
(..031)
Diff backup  (..031) DB1
T-7
(..031)
Full backup  (..045) FB2
T-6
Not restored  (..031)
Log backup  (..045) LB3
T-5
(..045)
Log backup  (..045) LB4
T-4
(..045)
Full backup  (..056) FB3
T-1
Not restored  (..45)

Assume no other backup taken apart from above backup and diff backup taken today.
We can restore diff backup (DF3) if

1.       FB3 backup not taken (FB1 + LB1+LB2+LB3+LB4) + DF3

2.       FB3 backup not taken (FB1 + LB1+LB2+LB3) + DF3

3.       FB3 backup not taken (FB2+ LB3+LB4) + DF3

4.       FB3 backup not taken (FB2+ LB3) + DF3

5.       FB3 backup not taken (FB2) + DF3

6.       FB3 backup taken but not restored DF3 won’t work with any combination and will be invalid until we restore the FB3.

what this means is that, as long as log backups are restored to the secondary, the Differential base LSN remains in step allowing you to bridge LSN gaps between the primary and the secondary databases using a differential backup.

SQL Browser service explaination


SQL Server browser service:
-------------------------------------------

The SQL Server Browser program runs as a Windows service. SQL Server Browser listens for incoming requests for Microsoft SQL Server resources and provides information about SQL Server instances installed on the computer. SQL Server Browser contributes to the following actions:
·         Browsing a list of available servers
·         Connecting to the correct server instance
·         Connecting to dedicated administrator connection (DAC) endpoints
 

TCP port 1433 and pipe \sql\query are assigned to the default instance, but those can be changed later by the server administrator using SQL Server Configuration Manager. Because only one instance of SQL Server can use a port or pipe, different port numbers and pipe names are assigned for named instances, including SQL Server Express.
If specific port is assigned client can connect with specifying por tno along with instance name but for dynamic port client won’t be aware of port no of instance. Here sql browser comes in picture to access the port no and instance name installed on server.

If the SQL Server Browser service is not running, you are still able to connect to SQL Server if you provide the correct port number or named pipe. For instance, you can connect to the default instance of SQL Server with TCP/IP if it is running on port 1433.
However, if the SQL Server Browser service is not running, the following connections do not work:

·         Any component that tries to connect to a named instance without fully specifying all the parameters (such as the TCP/IP port or named pipe).
·       Any component that generates or passes server\instance information that could later be used by other components to reconnect.
·         Connecting to a named instance without providing the port number or pipe.

·         DAC to a named instance or the default instance if not using TCP/IP port 1433.

If you are using SQL Server in a client-server scenario (for example, when your application is accessing SQL Server across a network), if you stop or disable the SQL Server Browser service, you must assign a specific port number to each instance and write your client application code to always use that port number. This approach has the following problems:

·         You must update and maintain client application code to ensure it is connecting to the proper port.
·         The port you choose for each instance may be used by another service or application on the server, causing the instance of SQL Server to be unavailable.

 

Network protocol in SQL Server


Network Protocol :
-------------------------
To connect to SQL Server Database Engine you must have a network protocol enabled. There are 4 types of protocol available with sql server.
1.       Shared memory
2.       TCP/IP
3.       Named pipes
4.       VIA
For any changes done in protocol properties requires sql server restart.
Shared memory :

Shared memory is the simplest protocol to use and has no configurable settings. Because clients using the shared memory protocol can only connect to a SQL Server instance running on the same computer, it is not useful for most database activity.
·         You cannot create an alias using the shared memory protocol.
·         if shared memory is enabled, then connecting to the Database Engine by name, creates a shared memory connection.
·         Connection name (except ip address) is same for TCP/IP(for remote client and local client) and shared memory(local client).If both enable local client always use shared memory.
·         Specifying an IP Address in the Server box will result in a TCP/IP connection.

The following names will connect to the local computer with the shared memory protocol if it is enabled:

·        
·         \
·         (local)
·         localhost

The following query will return the protocol used for the current connection.
 
SELECT net_transport FROM sys.dm_exec_connections WHERE session_id = @@SPID;
 
TCP/IP : (Transmission Control Protocol/Internet Protocol)

TCP/IP is a common protocol widely used over the Internet. For TCP/IP Sockets, data transmissions are more streamlined and have less overhead. Data transmissions can also take advantage of TCP/IP Sockets performance enhancement mechanisms such as windowing, delayed acknowledgements, and so on. This can be very helpful in a slow network. Depending on the type of applications, such performance differences can be significant.

At the time of connection, the SQL Server Native Client component reads the server, protocol, and port values from the registry for the specified alias name, and creates a connection string in the format tcp:[\], or tcp:[\],.

To create a valid connection string using TCP/IP, you must:
  • Specify an Alias Name.
  • For the Server, enter either a server name to which you can connect using the PING utility, or an IP address to which you can connect using the PING utility. For a named instance append the instance name.
  • Specify TCP/IP for the Protocol.
  • Optionally, enter a port number for the Port No. The default is 1433, which is the port number of the default instance of the Database Engine on a server. To connect to a named instance or a default instance that is not listening on port 1433, you must provide the port number, or start the SQL Server Browser service.

At the time of connection, the SQL Server Native Client component reads the server, protocol, and port values from the registry for the specified alias name, and creates a connection string in the format tcp:[\], or tcp:[\],.

 TCP/IP parameters:
·         Enabled  : Possible values are Yes and No.
·         Keep Alive : Specify the interval (milliseconds) in which keep-alive packets are transmitted to verify that the computer at the remote end of a connection is still available.
·         Listen All : Specify whether SQL Server will listen on all the IP addresses that are bound to network cards on the computer. If set to No, configure each IP address separately using the properties dialog box for each IP address. If set to Yes, the settings of the IPAll properties box will apply to all IP addresses. Default value is Yes.
·         No Delay : SQL Server does not implement changes to this property.
·         TCP Dynamic Ports  :Blank, if dynamic ports are not enabled. To use dynamic ports, set to 0.
For IPAll, displays the port number of the dynamic port used.
 

The default instance of SQL Server listens for incoming connections on port 1433. The port can be changed for security reasons or because of a client application requirement. By default, named instances (including SQL Server Express) are configured to listen on dynamic ports. To configure a static port, leave the TCP Dynamic Ports box blank and provide an available port number in the TCP Port box.

SQL Server Database Engine can listen on multiple ports on the same IP address, list the ports, separated by commas, in the format 1433,1500,1501. This field is limited to 2047 characters.

To configure a single IP address to listen on multiple ports, the Listen All parameter must also be set to No, on the Protocols Tab of the TCP/IP Properties

Since the dynamic port number can change each time SQL Server starts, SQL Server provides the SQL Server Browser Service, to monitor the ports, and direct incoming connections to the current port for that instance. Using dynamic ports complicates connecting SQL Server through a firewall because the port number may change when SQL Server is restarted, requiring changes to the firewall settings. To avoid connection problems through a firewall, configure SQL Server to use a static port.

 
Named Pipes:

Named Pipes is a protocol developed for local area networks. A part of memory is used by one process to pass information to another process, so that the output of one is the input of the other. The second process can be local (on the same computer as the first) or remote (on a networked computer).

For named pipes, network communications are typically more interactive. A peer does not send data until another peer asks for it using a read command. A network read typically involves a series of peek named pipes messages before it starts to read the data. These can be very costly in a slow network and cause excessive network traffic, which in turn affects other network clients.

Unless changed by the user, when the default instance of Microsoft SQL Server listens on the named pipes protocol, it uses \\.\pipe\sql\query as the pipe name. The period indicates that the computer is the local computer, pipe indicates that the connection is a named pipe, and sql\query is the name of the pipe. To connect to the default pipe, the alias must have \\\pipe\sql\query as the pipe name. If SQL Server has been configured to listen on a different pipe, the pipe name must use that pipe. For instance, if SQL Server is using \\.\pipe\unit\app as the pipe, the alias must use \\\pipe\unit\app as the pipe name.

To create a valid pipe name, you must:

  • Specify an Alias Name.
  • Select Named Pipes as the Protocol.
  • Enter the Pipe Name. Alternatively, you can leave Pipe Name blank and SQL Server Configuration Manager will complete the appropriate pipe name after you specify the Protocol and Server
  • Specify a Server. For a named instance you can provide a server name and instance name.

At the time of connection, the SQL Server Native Client component reads the server, protocol, and pipe name values from the registry for the specified alias name, and creates a pipe name in the format np:\\\pipe\ or np:\\\pipe\.

 For a named instance, the default pipe name is \\\pipe\MSSQL$\sql\query.

Generally, TCP/IP is preferred in a slow LAN, WAN, or dial-up network, whereas named pipes can be a better choice when network speed is not the issue, as it offers more functionality, ease of use, and configuration options

VIA:

Virtual Interface Adapter (VIA) protocol works with VIA hardware. For information about how to use VIA, contact your hardware vendor.

 
The VIA protocol is deprecated. This feature will be removed in a future version of Microsoft SQL Server. Avoid using this feature in new development work, and plan to modify applications that currently use this feature.

 

Friday, March 28, 2014

Understanding of CPU and thread process


Thread:

--------------

Worker  threads are the agents of SQL Server which are scheduled in CPU and they carry out the tasks,We can decide the no of threads using MAX_worker_threads option.

A value 0 means it can decide its max thread using below formula:

For 32 bit operating system:

Total available logical CPU’s <= 4 :     max worker threads = 256

Total available logical CPU’s > 4 :        max worker threads = 256 + ((logical CPUS’s - 4) * 8)

For 64 bit operating system:

Total available logical CPU’s <= 4 :     max worker threads = 512

Total available logical CPU’s > 4 :        max worker threads = 512 + ((logical CPUS’s - 4) * 16)

If you have set Max. Worker thread to 0, you can check the worker thread count calculated by SQL Server using the query

 SELECT max_workers_count FROM sys.dm_os_sys_info

When a request is received, SQL Server will try to locate a idle thread to schedule the request. If an idle worker was not located, new worker will be created. If all workers have been created and none is idle, then the request is queued which will be picked up by any worker thread from the pool of worker threads created. Query below gives the amount of worker threads created at the moment in your SQL Server:

SELECT  SUM(current_workers_count) as [Current worker thread] FROM sys.dm_os_schedulers

One of the main reason to to limit the max_worker_thread count to limit the resource used by each thread. In situations where the number of running tasks is less than  the defined number of threads, each task will have its own dedicated thread. If the number of concurrent tasks rises beyond the number of threads, threads are pooled, and pending tasks are granted threads from the pool when available.

 

AFFINITY MASK:

-----------------

 

SQL Server can be configured so that threads will only be assigned to particular CPUs. This setting is typically used in NUMA-enabled systems, or on systems used for environments where a certain level of CPU resource needs to be reserved for other purposes, for example, other SQL Server instances or applications, to prevent one SQL Server instance from dominating CPU usage.

For a single instance server it is best suited to leave the setting as default, but for multi instance server it should be customized.

There are two types of AFFINITY mask:

Processor Affinity Mask and I/O Affinity Mask

In the default configuration these two options are checked. That means that all the CPUs will be used by SQL processor and I/O modules.

 

Physical core  vs Logical core

---------------------------------------

 

To see how many cores are being used use below options:

1. See the startup message in error log for eg:

--using 40 logical processors based on SQL Server licensing. This is an informational message; no user action is required.

2. select cpu_count from sys.dm_os_sys_info

3.Back to counting CPUs. To find out how many CPUs a particular instance is actually using, run the following query which is based on the sys.dm_os_schedulers DMV:

select scheduler_id,cpu_id, status, is_online from sys.dm_os_schedulers where status='VISIBLE ONLINE'

 

Physical socket : on a motherboard where a physical processor fits (used for licensing)

Physical core : within a physical processor (multi-core)

Logical core : within a physical core (hyper-threading)

Hyper-threading creates two “logical processors” within each physical processor core of an actual physical processor, that are visible to the operating system

 

 

Remember that Windows and SQL Server cannot tell the difference between physical and logical cores. Running the query below will tell you how many logical cores are



visible and how many physical CPUs you have.

SELECT cpu_count AS [Logical CPU Count], hyperthread_ratio AS [Hyperthread Ratio], cpu_count/hyperthread_ratio AS [Physical CPU Count], physical_memory_in_bytes/1048576 AS [Physical Memory (MB)], sqlserver_start_time  FROM sys.dm_os_sys_info;

 

 

Maximum Degree of Parallelism (MAXDOP)

-----------------------------------------------------------

 

A commonly altered setting is Maximum Degree of Parallelism(MAXDOP), which controls the maximum number of CPUs that can be used in executing a single task or query. For example, a large query may be broken up into different parts, with each part executing threads on separate CPUs. Such a query is known as a parallel query.

By default, MAXDOP is 0, meaning that SQL Server is left to decide the appropriate number of CPUs to use. You can set this value to 1, effectively disabling parallel queries, or to a specific number that limits the number of CPUs that can be used.

In OLTP systems, use a maximum MAXDOP setting of 8, including systems with access to more than 8 CPU cores. The effort to split and rejoin a query across more than 8 CPUs often outweighs the benefits of parallelism

One of the downsides from setting MAXDOP to 1 is that certain operations, such as index rebuilds, benefit greatly from parallelism but are unable to do so with a



MAXDOP 1 setting. In such cases, you can specify the MAXDOP setting at a statement level. For example, the CREATE INDEX command, an example of which is shown here, accepts a MAXDOP parameter:

-- Use a MAXDOP hint to override the default server MAXDOP setting

CREATE NONCLUSTERED INDEX [IX_Address_StateProvinceID]    ON [Person].[Address] ([StateProvinceID] ASC)WITH (MAXDOP=0)GO

 

In this example, we specify MAXDOP = 0 to override the instance default MAXDOP setting, and thereby permit the index creation to be parallelized if SQL Server decides that's the best approach.

 

Best practice :

•For servers that use more than eight processors, use the following configuration:

MAXDOP=8

•For servers that use eight or fewer processors, use the following configuration:

MAXDOP=0 to N

Note In this configuration, N represents the number of processors.

•For servers that have NUMA configured, MAXDOP should not exceed the number of CPUs that are assigned to each NUMA node.

•For servers that have hyperthreading enabled, the MAXDOP value should not exceed the number of physical processors.

•For servers that have NUMA configured and hyperthreading enabled, the MAXDOP value should not exceed number of physical processors per NUMA node.

Wednesday, October 9, 2013

Attach database

sp_attach_db [ @dbname= ] N'SharePoint2010_Config',
@filename1=N'G:\LDF\SharePoint2010_Config.mdf',
@filename2=N'G:\SharePoint2010_Config_log.LDF'
GO
 
 The sp_attach_db stored procedure should only be executed on databases that were previously detached from the database server by using an explicit sp_detach_db operation or on copied databases. If you have to specify more than 16 files, use CREATE DATABASE database_name FOR
ATTACH or CREATE DATABASE database_name FOR_ATTACH_REBUILD_LOG

When you attach a replicated database that was copied instead of being detached, consider the following:
•If you attach the database to the same server instance and version as the original database, no additional steps are required.
•If you attach the database to the same server instance but with an upgraded version, you must execute sp_vupgrade_replication to upgrade replication after the attach operation is complete.
•If you attach the database to a different server instance, regardless of version, you must execute sp_removedbreplication to remove replication after the attach operation is complete.
•The source database must be at least version 80 (SQL Server 2000) to attach to SQL Server 2008. SQL Server 2000 or SQL Server 2005 databases that have a compatibility level less than 80 will be set to compatibility 80 when attached.

Attach using create database command  =>
USE master;
GO
--Detach the AdventureWorks database
sp_detach_db AdventureWorks;
GO
-- Physically move the full text catalog to the new location.
--Attach the AdventureWorks database and specify the new location of the full-text catalog.
CREATE DATABASE AdventureWorks ON
    (FILENAME = 'c:\Program Files\Microsoft SQL Server\MSSQL.1\MSSQL\Data\AdventureWorks_Data.mdf'),
    (FILENAME = 'c:\Program Files\Microsoft SQL Server\MSSQL.1\MSSQL\Data\AdventureWorks_log.ldf'),
    (FILENAME = 'c:\myFTCatalogs\AdvWksFtCat')
FOR ATTACH;
GO

To rebuild log file

USE [master]
GO
CREATE DATABASE [AdventureWorks] 
ON  PRIMARY ( NAME = N'AdventureWorks_Data',FILENAME = N'C:\SQLDBs\Data\RecoveryModelTesting_Data.mdf')
FOR ATTACH_REBUILD_LOG
GO

Tuesday, September 17, 2013

FTP command to exchange files between windows and hpux

E:\Script> ftp box.domain.co.uk

User (box.domain.co.uk:(none)): oracle
Password:

ftp>cd /home/test_script/script_original
pwd
257 "/home/test_script/script_original" is current directory.

ftp>help
Commands may be abbreviated.  Commands are:
!               delete          literal         prompt          send
?               debug           ls              put             status
append          dir             mdelete         pwd             trace
ascii           disconnect      mdir            quit            type
bell            get             mget            quote           user
binary          glob            mkdir           recv            verbose
bye             hash            mls             remotehelp
cd              help            mput            rename
close           lcd             open            rmdir

set to binary mode for transfering file to hpux from windows:
ftp>bi
200 Type set to I.
mput sa_readme.txt
mput sa_readme.txt? y

ftp>ascii
200 Type set to A.
mget sa_readme.txt
mput sa_readme.txt? y

ftp>bye