Oracle DBA – A lifelong learning experience

Controlling RMAN channels on RAC

Posted by John Hallas on December 3, 2014

This was sent to me for posting  by my friend and ex-colleague Vitaly Kaminsky ….

I have recently worked with the customer where standard RMAN backups of production 2-node cluster (11.2.0.3) were getting too big and taking longer than 24 hours to run.

The problem with this particular cluster was the fact that ALLOCATION of RMAN connections to the instances of the cluster was controlled by SCAN and driven by the load-balancing algorithm of SCAN.

The result was rather undesirable allocation of all channels to a single instance and the only way to control it was to reduce the workload on all nodes at the time when backup is started which was unacceptable on this 24/7 constant workload system.

This single-node allocation caused the network connections (10GB network) to become saturated, throttling the backup rates and degrading the “user experience” for user sessions on the same node.

After considerable search, I found a way of controlling this allocation I did not know about before (just showing my lack of attention when reading low-level manuals).

The way RMAN is using parameters is as follows:

  1. Use the instructions from the script
  2. If there are no script instructions, use the defaults.

How to force RMAN to “distribute” the channels across instances:

  1. Set required degree of parallelism in default parameters:

 

CONFIGURE DEVICE TYPE DISK PARALLELISM 4 BACKUP TYPE TO BACKUPSET;

RMAN> show all

CONFIGURE DEVICE TYPE DISK PARALLELISM 4 BACKUP TYPE TO BACKUPSET;
  1. “Hard-code” the channel allocation in the defaults:
CONFIGURE CHANNEL 1 DEVICE TYPE DISK CONNECT='sys/password@node1:1521/clusterservice' FORMAT='/backup/FULL/%d_%U';

CONFIGURE CHANNEL 2 DEVICE TYPE DISK CONNECT='sys/password@node1:1521/clusterservice' FORMAT='/backup/FULL/%d_%U';

CONFIGURE CHANNEL 3 DEVICE TYPE DISK CONNECT='sys/password@node2:1521/clusterservice' FORMAT='/backup/FULL/%d_%U';

CONFIGURE CHANNEL 4 DEVICE TYPE DISK CONNECT='sys/password@node2:1521/clusterservice' FORMAT='/backup/FULL/%d_%U';
  1. DO NOT allocate channels in the script:
RMAN> run {

2> backup as COMPRESSED BACKUPSET tag ‘FULL_112014100000' database;

3> }
  1. Check how channels are allocated:
RMAN> backup as COMPRESSED BACKUPSET tag ‘FULL_112014100000' database;

Starting backup at 24-NOV-14

allocated channel: ORA_DISK_1

channel ORA_DISK_1: SID=871 instance=racservice1 device type=DISK

allocated channel: ORA_DISK_2

channel ORA_DISK_2: SID=970 instance=racservice1 device type=DISK

allocated channel: ORA_DISK_3

channel ORA_DISK_3: SID=1453 instance=racservice2 device type=DISK

allocated channel: ORA_DISK_4

channel ORA_DISK_4: SID=399 instance=racservice2 device type=DISK

channel ORA_DISK_1: starting compressed full datafile backup set

 

There are 2 issues I can think of with this approach:

I am not keen on passing the passwords to any services, but RMAN is smart enough to cater for this:

RMAN> show all;

using target database control file instead of recovery catalog

RMAN configuration parameters for database with db_unique_name RACSERVICE are:

CONFIGURE DEVICE TYPE DISK PARALLELISM 4 BACKUP TYPE TO BACKUPSET;

CONFIGURE CHANNEL 1 DEVICE TYPE DISK CONNECT '*' FORMAT   '/backup/FULL/%d_%U';

CONFIGURE CHANNEL 2 DEVICE TYPE DISK CONNECT '*' FORMAT   '/backup/FULL/%d_%U';

CONFIGURE CHANNEL 3 DEVICE TYPE DISK CONNECT '*' FORMAT   '/backup/FULL/%d_%U';

CONFIGURE CHANNEL 4 DEVICE TYPE DISK CONNECT '*' FORMAT   '/backup/FULL/%d_%U';
  1. Correcting mistakes.

While typing, I made some mistakes and ended up with the wrong CONFIGURE CHANNEL lines.

How do you delete those lines?  Simples:

CONFIGURE CHANNEL 1 DEVICE TYPE DISK CLEAR;

The result of the above?

We now have even distribution of RMAN workload and much higher utilisation of available network bandwidth on both nodes!

 

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

 
%d bloggers like this: