ORA-19909: datafile 1 belongs to an orphan incarnation

Recovery Slave PR00 previously exited with exception 19909
Step 1 Recovery Slave PR00 previously exited with exception 19909

Case 

When we want to start the managed recovery [MRP] in the stand by if we are getting the the message in alert log and we are not able to start the MRP process in the stand by database 

Step 1 starting the redo apply in the Physical standby database 

ALTER DATABASE RECOVER MANAGED STANDBY DATABASE DISCONNECT

Attempt to start background Managed Standby Recovery process (OMTST1)

Wed May 25 06:57:12 2016

MRP0 started with pid=60, OS id=18711

MRP0: Background Managed Standby Recovery process started (OMTST1)

 started logmerger process

Wed May 25 06:57:17 2016

Managed Standby Recovery not using Real Time Apply

Wed May 25 06:57:17 2016

Warning: Recovery target destination is in a sibling branch

of the controlfile checkpoint. Recovery will only recover

changes to datafiles.

Datafile 1 (ckpscn 67283504852) is orphaned on incarnation#=2

MRP0: Detected orphaned datafiles!

Recovery will possibly be retried after flashback...

Errors in file /prod02/oracle/oratst/diag/diag/rdbms/omtst/OMTST1/trace/OMTST1_pr00_18715.trc:

ORA-19909: datafile 1 belongs to an orphan incarnation

ORA-01110: data file 1: '+DATA/omtst/datafile/system.806.911262335'

Recovery Slave PR00 previously exited with exception 19909

The MRP exist' with the error and not able to start the redo apply process .. and it report's the problem is with the rman incarnation

Step 2 list the incarnation form both the database

Primary database 

RMAN> list incarnation;

using target database control file instead of recovery catalog

List of Database Incarnations

DB Key  Inc Key DB Name  DB ID            STATUS  Reset SCN  Reset Time

------- ------- -------- ---------------- --- ---------- ----------

1       1       OMPROD   3907334877       PARENT  23900323136 12-DEC-13

2       2       OMPROD   3907334877       CURRENT 38315839033 31-DEC-14

DR  Stand by Database 

RMAN> list incarnation of database;

using target database control file instead of recovery catalog

List of Database Incarnations

DB Key  Inc Key DB Name  DB ID            STATUS  Reset SCN  Reset Time

------- ------- -------- ---------------- --- ---------- ----------

1       1       OMPROD   3907334877       PARENT  23900323136 12-DEC-13

2       2       OMPROD   3907334877       PARENT  38315839033 31-DEC-14

3       3       OMPROD   3907334877       CURRENT 66684359815 17-MAY-16

now you can find the Primary and stand by have different incarnation in the control file  so we can  reset the incarnation to same in the PROD  before we start the MRP process

Step 3 Change the incarnation to the match the primary database and both the incarnations should be same 

IN the Stand by database we need to change the rman incarnation to match as same as the PROD primary incarnation to start the recovery process the new incarnation which is set and update in the control file will not allow to start the MRP process until we reset the incarnation to as same in Primary

RMAN> list incarnation of database;

using target database control file instead of recovery catalog

List of Database Incarnations

DB Key  Inc Key DB Name  DB ID            STATUS  Reset SCN  Reset Time

------- ------- -------- ---------------- --- ---------- ----------

1       1       OMPROD   3907334877       PARENT  23900323136 12-DEC-13

2       2       OMPROD   3907334877       PARENT  38315839033 31-DEC-14

3       3       OMPROD   3907334877       CURRENT 66684359815 17-MAY-16 -- new incarnation created after restore incremental from primary 

RMAN> reset database  to incarnation 2;

database reset to incarnation 2

RMAN>  list incarnation of database;

using target database control file instead of recovery catalog

List of Database Incarnations

DB Key  Inc Key DB Name  DB ID            STATUS  Reset SCN  Reset Time

------- ------- -------- ---------------- --- ---------- ----------

1       1       OMPROD   3907334877       PARENT  23900323136 12-DEC-13

2       2       OMPROD   3907334877       CURRENT 38315839033 31-DEC-14

3       3       OMPROD   3907334877       ORPHAN  66684359815 17-MAY-16

once the incarnation in both the control file  are same then start the MRP process 

Step 4 Start the MRP process in the physical stand by

ALTER DATABASE RECOVER MANAGED STANDBY DATABASE DISCONNECT

Recovery Slave PR00 previously exited with exception 19909

Completed:  ALTER DATABASE RECOVER MANAGED STANDBY DATABASE DISCONNECT

Wed May 25 07:04:00 2016

MRP0: Background Media Recovery process shutdown (OMTST1)

Wed May 25 07:13:26 2016

Setting recovery target incarnation to 2

Wed May 25 07:16:14 2016

ALTER DATABASE RECOVER MANAGED STANDBY DATABASE DISCONNECT

Attempt to start background Managed Standby Recovery process (OMTST1)

Wed May 25 07:16:14 2016

MRP0 started with pid=56, OS id=19646

MRP0: Background Managed Standby Recovery process started (OMTST1)

 started logmerger process

Wed May 25 07:16:19 2016

Managed Standby Recovery not using Real Time Apply

Parallel Media Recovery started with 4 slaves

Wed May 25 07:16:20 2016

Warning: recovery process PR01 cannot use async I/O

Wed May 25 07:16:20 2016

Warning: recovery process PR02 cannot use async I/O

Wed May 25 07:16:20 2016

Warning: recovery process PR03 cannot use async I/O

Wed May 25 07:16:21 2016

Warning: recovery process PR04 cannot use async I/O

check  the alert long you can see the the MRP process had been started ..