[opmn] [ERROR:1] [] [libopmncustom] Process Ping Failed

[opmn] [ERROR:1] [] [libopmncustom] Process Ping Failed

When we are trying to start the Opmn process it gives this kind of error for one process .. it is due to the states file which holds the process id of the particular process and it states.. if Any changes are done in the file while the process is up then the when we bring down and restart the opmn process it will hold the changed state which will reflect the wrong Change

Step 1

Start the opmn process
[orabi@ob]$ ./opmnctl startall
opmnctl startall: starting opmn and all managed processes...
================================================================================
opmn id=obitst.oasiserp.com:9501
Response: 4 of 5 processes started.

ias-instance id=instance1
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
ias-component/process-type/process-set:
  coreapplication_obips1/OracleBIPresentationServicesComponent/coreapplication_obips1/

Error
--> Process (index=1,uid=1374230127,pid=8063)
  failed to start a managed process after the maximum retry limit
  Log:
  <base_dir>/fmw/instances/instance1/diagnostics/logs/OracleBIPresentationServicesComponent/coreapplication_obips1/console~coreapplication_obips1~1.log

Step2
check the log files of the opmn to find the exact cause of the issue
[bi@ob opmn]$ pwd
<base_dir>/fmw/instances/instance1/diagnostics/logs/OPMN/opmn
[bi@ob opmn]$ ll
total 668
-rw-r----- 1 orabi oinstall  39960 May  1 17:25 logquery~1.log
-rw-r----- 1 orabi oinstall  39533 May  1 17:25 logquery~2.log
-rw-r----- 1 orabi oinstall  37385 May  1 17:25 logquery~3.log
-rw-r----- 1 orabi oinstall  37374 May  1 17:25 logquery~4.log
-rw-r----- 1 orabi oinstall  37925 May  1 17:25 logquery~5.log
-rw-r----- 1 orabi oinstall 467588 May  2 13:13 opmn.log
-rw-r----- 1 orabi oinstall   1632 May  2 13:12 opmn.out

Step 3
Open the opmn.log file and you will find the error messages like below .. the particular uid which is stored in the states folder will not be changed Science the new change is not acceptable 

[2013-05-02T13:13:00+04:00] [opmn] [ERROR:1] [] [libopmncustom] Ping failed in Ready callback for proc:1374230126
[2013-05-02T13:13:02+04:00] [opmn] [ERROR:1] [] [libopmncustom] Process Ping Failed: coreapplication_obis1~OracleBIServerComponent~coreapplication_obis1~1 (1374230126:7905) [No addresses matched the properties for the request]
[2013-05-02T13:13:02+04:00] [opmn] [ERROR:1] [] [libopmncustom] Process Ping Failed: coreapplication_obis1~OracleBIServerComponent~coreapplication_obis1~1 (1374230126:7905)
[2013-05-02T13:13:02+04:00] [opmn] [ERROR:1] [] [libopmncustom] Ping failed in Ready callback for proc:1374230126
[2013-05-02T13:13:04+04:00] [opmn] [ERROR:1] [] [libopmncustom] Process Ping Failed: coreapplication_obis1~OracleBIServerComponent~coreapplication_obis1~1 (1374230126:7905) [No addresses matched the properties for the request]
[2013-05-02T13:13:04+04:00] [opmn] [ERROR:1] [] [libopmncustom] Process Ping Failed: coreapplication_obis1~OracleBIServerComponent~coreapplication_obis1~1 (1374230126:7905)
[2013-05-02T13:13:04+04:00] [opmn] [ERROR:1] [] [libopmncustom] Ping failed in Ready callback for proc:1374230126
[2013-05-02T13:13:06+04:00] [opmn] [NOTIFICATION:1] [665] [pm-process] Process Alive: coreapplication_obis1~OracleBIServerComponent~coreapplication_obis1~1 (1374230126:7905)
[2013-05-02T13:13:06+04:00] [opmn] [NOTIFICATION:1] [668] [pm-requests] Request 2 Completed. Command: /start
                         
Step 4
Navigate to the states file location and find the states file and rename the states file into the new one when the opmn start it will create the new states file which will hold the current details in it

[bi@ob opmn]$ pwd
<base_dir>/fmw/instances/instance1/config/OPMN/opmn
[bi@ob opmn]$ ls
instance.properties  instance.properties.bak  opmn.xml  ports.prop  states  states742013  states_orginal  wallet
[bi@ob opmn]$ cd states
[bi@ob states]$ ls
p1374230126  p1374230128  p1374230129  p1374230130
[orabi@obitst states]$ ll
total 16
-rw-r----- 1 orabi oinstall 487 May  2 13:13 p1374230126
-rw-r----- 1 orabi oinstall 264 May  2 13:12 p1374230128
-rw-r----- 1 orabi oinstall 508 May  2 13:12 p1374230129
-rw-r----- 1 orabi oinstall 240 May  2 13:12 p1374230130

Step5

If any one of the particular process in the opmn is not Coming up then check the log file for that Process to find the exact reason behind it

[bi@ob bin]$ ./opmnctl status -l
Processes in Instance: instance1
---------------------------------+--------------------+---------+----------+------------+----------+-----------+------
ias-component                    | process-type       |     pid | status   |        uid |  memused |    uptime | ports
---------------------------------+--------------------+---------+----------+------------+----------+-----------+------
coreapplication_obiccs1          | OracleBIClusterCo~ |   15448 | Alive    |  882845249 |   200772 |   0:02:07 | N/A
coreapplication_obisch1          | OracleBIScheduler~ |   15446 | Alive    |  882845248 |   403024 |   0:02:07 | N/A
coreapplication_obijh1           | OracleBIJavaHostC~ |   15447 | Alive    |  882845247 |  1465000 |   0:02:07 | javahost:9810
coreapplication_obips1           | OracleBIPresentat~ |     N/A | Down     |        N/A |      N/A |       N/A | N/A
coreapplication_obis1            | OracleBIServerCom~ |   15444 | Alive    |  882845245 |  1318908 |   0:02:07 | N/A
Try to start the particular process and check the log files and fix the issue

[bi@ob bin]$ ./opmnctl startproc ias-component=coreapplication_obips1
opmnctl startproc: starting opmn managed processes...
================================================================================
opmn id=obitst.oasiserp.com:9501
Response: 0 of 1 processes started.

ias-instance id=instance1
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
--------------------------------------------------------------------------------
ias-component/process-type/process-set:
  coreapplication_obips1/OracleBIPresentationServicesComponent/coreapplication_obips1/

Error
--> Process (index=1,uid=882845250,pid=15806)
  failed to start a managed process after the maximum retry limit
  Log:
  <base_dir>/fmw/instances/instance1/diagnostics/logs/OracleBIPresentationServicesComponent/coreapplication_obips1/console~coreapplication_obips1~1.log

Navigate to the particular log folder for that particular  service then check the log files for it
[orabi@obitst coreapplication_obips1]$ pwd
<base_dir>/fmw/instances/instance1/diagnostics/logs/OracleBIPresentationServicesComponent/coreapplication_obips1

[2013-05-02T13:32:35.000+04:00] [OBIPS] [ERROR:31] [] [saw.i18n.currencyLookup.loadCurrencies] [ecid: ] [tid: ] Sax parser returned an exception.
Message: An exception occurred! Type:UTFDataFormatException, Message:invalid byte 1 () of a 1-byte sequence., Entity publicId: <base_dir>/fmw/Oracle_BI1/bifoundation/web/display/currencies.xml, Entity systemId: , Line number: 1, Column number: 40[[
File:saxreader.cpp
Line:681
Location:
        saw.i18n.currencyLookup.loadCurrencies
        saw.webextensionbase.init
        saw.sawserver
currenciesFile: <base_dir>/fmw/Oracle_BI1/bifoundation/web/display/currencies.xml


Step 6
Once you remove the states folder and the fixed any other issue causing the failure then conform that no opmn process is currently running and start the opmnctl process
Navigate to the <base_dir>/fmw/instances/instance1/bin
[bi@ob bin]$ ./opmnctl status -l

Processes in Instance: instance1
---------------------------------+--------------------+---------+----------+------------+----------+-----------+------
ias-component                    | process-type       |     pid | status   |        uid |  memused |    uptime | ports
---------------------------------+--------------------+---------+----------+------------+----------+-----------+------
coreapplication_obiccs1          | OracleBIClusterCo~ |    9801 | Alive    |  882845234 |   201904 |   0:05:47 | N/A
coreapplication_obisch1          | OracleBIScheduler~ |    9799 | Alive    |  882845233 |   337500 |   0:05:47 | N/A
coreapplication_obijh1           | OracleBIJavaHostC~ |    9800 | Alive    |  882845232 |  1466316 |   0:05:47 | javahost:9810
coreapplication_obips1           | OracleBIPresentat~ |    9798 | Alive    |  882845231 |   491884 |   0:05:47 | N/A
coreapplication_obis1            | OracleBIServerCom~ |    9797 | Alive    |  882845230 |  1316932 |   0:05:47 | N/A