Novell acquired - Unix status uncertain

Oh, dear! Wonder where that is going to end. Not another SCO-like event, I hope.

And, in the press: MicroSoft bought "intelectual properties" from Novell. Which properties that would be is unclear.

Update: Novell released this: Novell will continue to own Novell’s UNIX copyrights following completion of the merger as a subsidiary of Attachmate.

ORA-12641 "Authentication service failed to initialize" on multi homed machines

Update: the TMPDIR environment variable gets picked up; the file will be created in that directory. A technical note will be released, as this is not documented anywhere.
I have been struggeling to get Kerberos Authentication running on multi homes machines at my clients site. I just could not figure out why one package (HP-speak for virtual machines) works just fine, where the next errs off with a dreaded "ORA-12641: Authentication service failed to initialize".
Mind you - both using the same keytab and Kerberos configuration files, and similar sqlnet.ora files.

Go trace, young man, go trace!

Today I had the awakening moment to enable server tracing on the database side: added the following code to sqlnet.ora:

trace_level_server = 32
trace_file_server = svr_trc
trace_directory_client = $TNS_ADMIN/trace

Sure enough, comparing the package with errors against the one that validated me just fine using Kerberos, gave me some idea:

[19-NOV-2010 12:46:38:931] nauk5ra_rcinit: Validating existing replay cache.
[19-NOV-2010 12:46:38:931] snauk5g_open_file: entry
[19-NOV-2010 12:46:38:931] snauk5g_open_file: Setting write lock.
[19-NOV-2010 12:46:38:931] snauk5g_open_file: Opening /var/tmp/kerb5srv.RC.
[19-NOV-2010 12:46:38:931] snauk5k_lock_file: entry
[19-NOV-2010 12:46:38:931] snauk5k_lock_file: Setting exclusive lock.
[19-NOV-2010 12:46:38:931] snauk5k_lock_file: exit
[19-NOV-2010 12:46:38:931] snauk5g_open_file: exit
[19-NOV-2010 12:46:38:931] nauk5rz_validate: entry
[19-NOV-2010 12:46:38:931] nauk5rz_validate: exit
[19-NOV-2010 12:46:38:931] snauk5t_close_file: entry
[19-NOV-2010 12:46:38:931] snauk5k_lock_file: entry
[19-NOV-2010 12:46:38:931] snauk5k_lock_file: Resetting lock.
[19-NOV-2010 12:46:38:931] snauk5k_lock_file: exit
[19-NOV-2010 12:46:38:931] snauk5t_close_file: exit

Now, compare that to the faulty environment:

[19-NOV-2010 12:38:56:805] nauk5ra_rcinit: Validating existing replay cache.
[19-NOV-2010 12:38:56:805] snauk5g_open_file: entry
[19-NOV-2010 12:38:56:805] snauk5g_open_file: Setting write lock.
[19-NOV-2010 12:38:56:805] snauk5g_open_file: Opening /var/tmp/kerb5srv.RC.
[19-NOV-2010 12:38:56:805] snauk5g_open_file: open failed with 13: File I/O error
.
[19-NOV-2010 12:38:56:805] snauk5g_open_file: Returning 203: File not found
.
[19-NOV-2010 12:38:56:805] snauk5g_open_file: exit
[19-NOV-2010 12:38:56:805] nauk5ra_rcinit: Could not open cache.
[19-NOV-2010 12:38:56:805] nauk5ru_create: entry
[19-NOV-2010 12:38:56:805] nauk5ru_create: Creating replay cache.
[19-NOV-2010 12:38:56:806] snauk5g_open_file: entry

This will eventually end in

[19-NOV-2010 12:38:56:807] na_csrd: exit
[19-NOV-2010 12:38:56:807] nacomer: error 12641 received from authentication service

Workaround

Now, I would not write this, if I did not need to remind myself of the following. There is a workaround until this is permanently fixed by Oracle, but it is not without risk.
The problem is on file permissions. Once the file is created, it gets 600, or read-write for owner only permissions. Change that to 666 (read-write to everybody), and you're done.
In fact, the file does not need to have any contents:

rm /var/tmp/kerb5srv.RC
touch /var/tmp/kerb5srv.RC
chmod 666 /var/tmp/kerb5srv.RC

Asof now, all packages on this machine can use Kerberos authentication.
Please note: kerb5srv is the name of the Service Principal chosen by my client overhere - your implementation may use a different name. Use that (or, trace to see what file is used)

By the way, the entry on Privacy was updated, due to the signing of this.

The Catch

Apart from the fact you may have reservations to open up a file to the world like that, the file itself resides in a temporary directory, /var/tmp.
If any system administrator decides to clean that up, the dreaded ORA-12641 will be all over the place.
Just rerun code above, and you're done.

Permanent Solution

There seems no sqlnet variable to cure this; sqlnet.KERBEROS5_CC_NAME brings nothing. Strictly speaking, it is not the Credential Cache, but the service principal cache, so that figures.
What I would like to see is one of:

• introduce an environment variable that allows to govern location of this file
• use a suffix, just like in the credentials cache.

Update: the TMPDIR environment variable does point one...

Congratulations, Sebastian

Sebastian Vettel, youngest world champion Formula 1 ever, congratulations!

And perhaps Ferrari should rethink contracting Mr. F. Alonso - the unsportsmanlike behavior towards Petrov says it all... If you cannot pass Petrov in like 20-odd rounds, you're clearly not World Champion material. There's only one to blame, Fernando - that is you. Not Petrov!

Happy Days

Some post are still in backlog, I've been here...

Support pays off!

Update on a previous entry: we have a patch, and it actually works.
So when you ever get in the situation where fixed user database link do not work due to kerberos aare environments, ask for patch 8482256.
Available (and tested!) for HP-UX 11.23 and up; should become available for MS Windows (Server 2003, 32 bit and up), as well as VMS.

mount Oracle

Just curious:

mountd 20048/tcp NFS mount protocol
mountd 20048/udp NFS mount protocol
# Nicolas Williams <Nicolas.Williams_at_oracle.com> 09 August 2010

ORA-12638: Status 82 "Not a bug"

One thing that completely spoils the previous entry on Kerberos authentication in the database, is the state of denial Oracle is in. It seems impossible to admit there's a bug in SQL*Net in the sense there's no fallback for username/password authentication when Kerberos fails.

ORA-12638 [Credential retrieval failed]

What's the case? Well, after you have set up Kerberos authentication, your database links seize working with ORA-12638: Credential retrieval failed, as soon as both databases are Kerberos aware.

Now, of course this is a situation you do not want. You want administrators and users to log in using their working environment credentials, but you rely on named accounts, and passwords, for database links.
Alas. Here comes Bug 6341668: CLIENT CAN NOT CONNECT TO DATABASE BY PASSWORD IF ANY KERBEROS PROBLEM OCCURS. Current status: 82 "Closed, not a bug". Al least someone else tried Kerberos authentication...
This is in contradiction with another Kerberos-related bug, # 2647883: Kerberos current user database links do not work (See Doc ID 2647883.8), where the actual workaround is ... use fixed user database links (!).
Catch-22, eh?

I hate it when I get confronted with this rubbish on Friday afternoons. It ruins my weekend!
Oh - might anyone wonder: this is cross platform, 10.2.0.4.

One other reference on Oracle Support: TNS-12638 when database authentication tried on Kerberos setup (Doc ID 972896.1)

Updates

ASO is standard

There's a note stating the Advanced Security Option is now standard with the Enterprise Database Install. Makes you wonder: is it included in the standard license fee, then?

Patch 10.2.0.5

Took the time to update the 10.2.0.4 environment I tested the scenario in to 10.2.0.5, which does have some Kerberos related bugs resolved. The result of all this is not only a ruined Sunday, it's also not resolving anything.

C:\>sqlplus /@db10
SQL*Plus: Release 10.2.0.5.0 - Production on Sun Sep 26 18:32:25 2010
Copyright (c) 1982, 2010, Oracle. All Rights Reserved.

Connected to:
Oracle Database 10g Enterprise Edition Release 10.2.0.5.0 - Production
With the Partitioning, Data Mining and Real Application Testing options

SQL> select sysdate from dual@fvb;
select sysdate from dual@fvb
*
ERROR at line 1:
ORA-12638: Credential retrieval failed

Only after removing the services from sqlnet.ora on the server side, it works:

[ora10@db1 admin]$ cat sqlnet.ora
names.default_domain=home.local
# sqlnet.authentication_services=(beq, kerberos5)
sqlnet.kerberos5_conf=/oracle/db10/admin/krb5.conf
sqlnet.kerberos5_keytab=/oracle/db10/admin/db1.keytab
sqlnet.authentication_kerberos5_service=orcl
sqlnet.kerberos5_conf_mit=true

Which makes the same query on the client work... instantaneous!

SQL> select sysdate from dual@fvb;

SYSDATE
---------
26-SEP-10

It's getting weirder

I stumbled upon this, trying to find a workable solution on maintainting two TNS_ADMIN locations on the server. A similar problem has been logged as Bug 5054469: "DATABASE AUTHENTICATED USERS REQUIRE KERBEROS CREDENTIAL CACHE TO CONNECT", which has been closed as duplicate of (internal) bug 9056628.
Explain: how can a bug# 5 million-and-something be closed as a duplicate of bug# almost-twice-as-high? Makes me wonder.
And the best is this: Doc ID 972896.1 (TNS-12638 when database authentication tried on Kerberos setup) states "This has been an issue in the past that the authentication services are not skipped to next when first fails".
Yeah right: has been... in the past...