SSO Fails randomly; clock skew too great

Ran against some weird issues today. At random people failed to pass Single Sign On. Not until I started to monitor the OC4J Security logging (in $ORACLE_HOME/opmn/logs) that I saw a failure

GSSException raised: Failure unspecified at GSS-API level (Mechanism level: Clock skew too great (37))

The Kerberos configuration file allows for a defined clock skew, and has a default of 300 (units: seconds).
There was no such entry in /etc/krb5, so the default was used.

Timestamps

Comparing system date&time against the timestamps written in the logs, I noticed indeed a difference of a little over 5 minutes. That explained the error, which was correct (too much time difference indicates a security breach, according to Kerberos)
Restarting the services resolved that, but this is not what you want in a 24x7 operation.

-XX:+UseGetTimeOfDay

Then, I recalled an issue with Java internal clocks slowly running out of sync with the operating system clock - seems specific for HP-UX, and can be resolved by specifying

-XX:+UseGetTimeOfDay

in the options section of the process at hand.

More specifics: the error code in full:

11/05/24 08:13:28 GSSException raised: Failure unspecified at GSS-API level (Mechanism level: Clock skew too great (37))
11/05/24 08:13:28 GSSException: Failure unspecified at GSS-API level (Mechanism level: Clock skew too great (37))
11/05/24 08:13:28 at sun.security.jgss.krb5.Krb5Context.acceptSecContext(Krb5Context.java:734)
11/05/24 08:13:28 at sun.security.jgss.GSSContextImpl.acceptSecContext(GSSContextImpl.java:300)
11/05/24 08:13:28 at sun.security.jgss.GSSContextImpl.acceptSecContext(GSSContextImpl.java:246)
11/05/24 08:13:28 at oracle.security.jazn.oc4j.KerberosAuthenticator.gssAuthenticate(Unknown Source)
11/05/24 08:13:28 at oracle.security.jazn.oc4j.KerberosAuthenticator.getAuthentication(Unknown Source)
11/05/24 08:13:28 at com.evermind.server.http.EvermindHttpServletRequest.getUserPrincipalInternal(EvermindHttpServletRequest.j
ava:3736)
11/05/24 08:13:28 at com.evermind.server.http.HttpApplication.authenticate(HttpApplication.java:6341)
11/05/24 08:13:28 at com.evermind.server.http.HttpApplication.getRequestDispatcher(HttpApplication.java:2872)
11/05/24 08:13:28 at com.evermind.server.http.HttpRequestHandler.processRequest(HttpRequestHandler.java:680)
11/05/24 08:13:28 at com.evermind.server.http.AJPRequestHandler.run(AJPRequestHandler.java:224)
11/05/24 08:13:28 at com.evermind.server.http.AJPRequestHandler.run(AJPRequestHandler.java:133)
11/05/24 08:13:28 at com.evermind.util.ReleasableResourcePooledExecutor$MyWorker.run(ReleasableResourcePooledExecutor.java:192
)
11/05/24 08:13:28 at java.lang.Thread.run(Thread.java:534)

A snippet from the opmn.xml file, after the change:

<module-data>
   <category id="start-parameters">
      <data id="java-options" value="-server -XX:+UseGetTimeOfDay -Djava.security.policy=/u2/oracleas/j2ee/OC4J_SECURITY/config/java2.policy -Djava.awt.headless=true -Xmx512m -Djavax.security.auth.useSubjectCredsOnly=false -Doracle.security.jazn.config=/u2/oracleas/j2ee/OC4J_SECURITY/config/jazn.xml

Remember to update all Oracle internals:

$ORACLE_HOME/dcm/bin/dcmctl updateconfig -ct opmn
$ORACLE_HOME/opmn/bin/opmnctl reload

Derived from Oracle note 352466.1 (which states the clock runs too fast...)