Sunday, January 17, 2010

Special Characters - notes and comments

Reason for this entry is the rather disturbing observation, you can actually get different results from one and the same script.
The scenario is running a script, that uses embedded "special" characters (an €, in this case) and is executed using either the CLI version of SQL*Plus (sqlplus.exe), or the Windowed version (sqlplusw.exe).

Code Pages

It turns out that both versions use different MS Windows code pages: the Windowed version, sqlplusw.exe, picks up the code page of MS Windows, while the CLI version uses the code page of the CLI. I ended up with different binary values of the character.
By default, these differ on all non-US installations of MS Windows!

Debugging

In the course of the blog entries, I have mentioned, but never made a list of tell-tale signs. A troubleshooting list so to say. Here's a start:
  • Remember: there's always at least one way to screw up. There's no recipe, no Rule-of-Thumb, or whatever to make sure you'll never end up with weird glyphs.
    Unless you control the whole, single-user, system, that is...
  • If you see a small diamond in web browser, start hitting your web designer/builder: the page encoding is incorrect for the characters used.
    You can change the Character Set Coding in most browsers as workaround.
  • If you see inverted ('"upside-down") question marks, your Oracle infrastructure is to blame: somewhere along the line, character set conversion takes place, for which the "to-character set" does not have a glyph defined (e.g. ISO8859P1 does not have a glyph for the Euro-sign)


[add Apr 2011] A rather good note on this is Doc ID 158577.1 on Metalink.

Wednesday, January 13, 2010

Ubuntu 9.10 (and Oracle's support for it)

It's official... CentOS is exit, I'm now running Ubuntu 9.10 (or Karmic Koala).

Of course, I'll set up Oracle again, which appears to be very easy, as Oracle provided a Debian (and thus Ubuntu) repository. This should allow Debian and Ubuntu users to install Oracle by using Synaptic Package Manager. Point an shoot installs... I am curious!

Add the repository

You do have to make Ubuntu aware of the fact Oracle is a source of software, too. For that, you need to add the location of the repository to your configuration. This is accomplished by:

frank@cs-frank03:~$ sudo gedit /etc/apt/sources.list
Scroll to the bottom of the file, and add:
# Oracle stuff
deb http://oss.oracle.com/debian unstable main non-free

After that, update the libraries to allow Software Center to show Oracle, enter:

frank@cs-frank03:~$ sudo apt-get update

Now, fire up Synaptic, and click on the 'Origin' button; you should see oss.oracle.com/main listed. Point at it, and you will see XE (in two flavours) and the XE client.

Now, if only I just wanted XE...

yum


For non-Debian, there's support for yum as well; check it out!

Tuesday, January 12, 2010

More APEX notes

A very, very quick way of housekeeping:

BEGIN
APEX_INSTANCE_ADMIN.REMOVE_WORKSPACE ('&workspace_name','Y','Y');
END;
/

It will drop your code, database accounts, and tablespace associated with it.

If you ever want to create a workspace, and do some basic housekeeping:

set echo off verify off
accept apex_short_name prompt 'Enter the short APEX applicaton name: '
prompt ...Creating the APEX tablespace...
--
create tablespace &apex_short_name
datafile '/oracle//&apex_short_name.01.dbf'
size 10m
autoextend on
next 5m
maxsize 1000m
extent management local
uniform size 64k;
-- User and grants
prompt ...Creating user and granting rights...
--
grant connect, resource
to &apex_short_name._owner identified by &apex_app_db_password;
grant execute on dbms_pipe to &apex_short_name._owner;
alter user &apex_short_name._owner default atblespace &apex_short_name;
alter user &apex_short_name._owner quota unlimited on &apex_short_name;
-- APEX Workspace and schema
prompt ...Creating APEX Workspace &apex_short_name
BEGIN
APEX_INSTANCE_ADMIN.ADD_WORKSPACE (
p_workspace => '&apex_short_name',
p_primary_schema => '&apex_short_name._owner',
p_additional_schemas => null);
END;
/
--
prompt ...Creating APEX Workspace administrator account...
--
begin
wwv_flow_api.set_security_group_id(
p_security_group_id=>APEX_UTIL.FIND_SECURITY_GROUP_ID('&apex_short_name'));
end;
/

begin
wwv_flow_fnd_user_api.create_fnd_user (
p_user_name => '&apex_short_name._ADMIN',
p_web_password => '&apex_short_name._ADMIN',
p_group_ids => '',
p_developer_privs=> 'ADMIN:CREATE:DATA_LOADER:EDIT:HELP:MONITOR:SQL',
p_default_schema=> '&apex_short_name._OWNER',
p_account_locked=> 'N',
p_account_expiry=> to_date('201001011300','YYYYMMDDHH24MI'),
p_failed_access_attempts=> 0,
p_change_password_on_first_use=> 'N',
p_first_password_use_occurred=> 'Y',
p_allow_access_to_schemas => '');
end;
/

commit;

For a given value of "abc" for apex_short_name, this will create:
  • a tablespace ABC, with an associated datafile abc01.dbf
  • a database account abc_owner with a password of your choice
  • an APEX workspace abc, with an associated abc_owner schema
  • an APEX Workspace administrator abc_ADMIN, with a password abc_ADMIN

Mind you: APEX Passwords are case sensitive; account names are not.

Monday, January 11, 2010

RSA-768 cracked

Today I read the announcement, RSA-768 is factorized, that is, the prime numbers of which the 232-digit number consists, are known.
English, and original, announcement, and a Dutch comment.

It may have taken a lot of time, but it just shows: nothing is secret. I previously blogged about SHA1 and MD-5 being broken, and actually being used for forgery.

Nice date format


Today is (was) 54 decimal.

Wednesday, January 06, 2010

Special characters - part V

The end of the War of the Worlds?


Well... that remains to be seen. I predict the conclusion will be that there's always a possibility unwanted, or "wrong" characters get displayed. But when you start to understand why, my mission is accomplished. By the way - I found an excellent explanation on HTML right here

Setup


For the setup, I used the "we8" database instance (see this entry) I added the 10.1.3.3 Oracle Http Server(based on Apache 2), of which Linux version used to be downloadable off the database download page. There was a bit of work to be done to get things running. The official documentation states:
For Apache 2.0 startup to succeed, link the current libdb.so version to /usr/lib/libdb-3.3.so.

For example, if the current shared library is in /usr/lib is libdb-4.1.so, then run the following command:

cd /usr/lib
ln -s /usr/lib/libdb-4.1.so libdb-3.3.so

Database Access Descriptor (DAD)


My DAD looks simple enough:

# ============================================================================
# mod_plsql DAD Configuration File
# ============================================================================
# 1. Please refer to dads.README for a description of this file
# ============================================================================
<Location /pls/myapp>
SetHandler pls_handler
Order deny,allow
Allow from all
AllowOverride None
PlsqlDatabaseConnectString cs-frank03:1521:we8
PlsqlAuthenticationMode Basic
PlsqlErrorStyle ModplsqlStyle
PlsqlNLSLanguage American_America.UTF8
PlsqlDefaultPage scott.home
</Location>

I added my own html wrapper, and created the following procedure check if the DAD was configured correctly:

create or replace procedure home as
begin
htp.p(format.pagestart(
p_title=>'DAD Check page',
p_text=>'DAD Checked out successfully!')
);
end;
/

This results in a "DAD Checked out successfully!" message, signifying the http server works, the connection to the database works, and my wrapper works.

Webcode


Knowing all the basics are working, I created the following package to display the contents of the blah table:

create or replace package demochar as
procedure showblah;
prodedure editblah;
end;
/
create or replace package body demochar
as
procedure showblah
is
begin
htp.p(format.pagestart(p_title => 'Demonstration of Special Characters on the Web',
p_text => 'Welcome - contents of table is')
);
htp.p('</div>');
htp.p('<div class="content">');
htp.p('<table summary="blah" border="1" align="center">');
htp.p('<tr><th>Character<th>Hex value');
for i in (select b.a, dump(b.a,16) as d
from blah b)
loop
htp.p('<tr>');
htp.p('<td>' || i.a);
htp.p('<td>' || i.d);
end loop;
htp.p('</table>');
htp.p(format.pageend);
end showblah;
procedure editblah is
begin
null;
end editblah;
begin
DEBUG.setlevel(2);
DEBUG.setdestination(DEBUG.destweb);
DBMS_RANDOM.initialize(TO_NUMBER(TO_CHAR(SYSDATE, 'miss')));
end;
/

Data

The blah table is there again, and it's contents is restored:

SQL> col dump(a,16) format a20
SQL> select a, dump(a,16) from blah;

A DUMP(A,16)
-------------------- --------------------
€ Typ=1 Len=1: 80
Æ Typ=1 Len=1: c6
ß Typ=1 Len=1: df

Results and observations


First of all, the UTF8, used in the DAD is standard APEX, to name one. But is it the correct one to use?
Displayed on the web, the page looks like this (Linux/Firefox):

Om MS Windows (XP) with Firefox, as well as using Konqueror on Linux, the dreaded diamond is displayed:


So - the fact that our character does not display in a web browser is depending on the browser and on what is used in the DAD. As we change the DAD characterset to be used to
PlsqlNLSLanguage American_America.WE8ISO8859P15
the page is displayed correctly in Konquerer, incorrect in Firefox on Linux, but that can be corrected by forcing the page to use the 1252 codepage (!). Firefox under MS Windows reacts the same: auto detection, Universal and ISO-8859-15 all do not work, changing the Character Encoding to West European(Windows 1252) displays the page correctly. Internet Explorer V7.0.5730.13: same story: auto detection does not work.

Now - the question arises: is this due to the coding of the page? Character sets are encoded in the header. The last screenshot is the same on all browsers/systems, and is not due to encoding settings, but purely Oracle - the DAD characterset is now WE8ISO8859P1, which indeed does not know how to map the € sign. That results in an upside-down question mark.


Windows...
After inserting the euro character in the MS Windows encoding (0xa4, or 164), all browsers manage to show correct values (that is the € sign) for that when using UTF8 in the DAD, as well as browser Character Encoding. Using WE9ISO8859P15 in the DAD, the pages will display one € correct, based on Character Encoding in the Browser:
- Western ISO-8859-15 will display € for 0xa4
- Western (Windows-1252) will display € for 0x80

Only Konqueror mangages to display the € symbol for both codes, when the DAD uses WE8ISO8859P15, and the page is explicitly coded in ISO-8859-15 (using the META element Content-type in the header):


Conclusion


There's alwyas at least one way to screw up. If it's not the database, that was fed the wrong code points, it's the browser, that does a lousy job. Konqueror does an admirable job, though. Internet Explorer as well as Firefox have the possibility to switch Character Coding, which may result in correct display of the glyphs.
UTF-8 is not always the best PlsqlNLSLanguage!

Friday, December 18, 2009

APEX Notes

If you want to access a procedure without going through the APEX security, alter wwv_flow_epg_include_mod_local to include your procedure.
I had a simple procedure, that lists all applications, called apex_apps:

CREATE OR REPLACE PROCEDURE apex_030200.apex_apps
AS
BEGIN
FOR i IN (SELECT TRIM(TO_CHAR (display_id)) ID, NAME
FROM apex_030200.wwv_flows
WHERE display_id NOT BETWEEN 4000 AND 5000
ORDER BY NAME)
LOOP
HTP.p ('<li><a href="f?p=' || i.ID || '">' || i.NAME || '</a></li>');
END LOOP;
END;


However, calling this resulted in:
[Fri Dec 18 15:57:12 2009] [error] [client x.x.x.x.] 
mod_plsql: /apex/apex_apps HTTP-403
It is forbidden to call this procedure directly from the browser!

After changing wwv_flow_epg_include_mod_local to

CREATE OR REPLACE function APEX_030200.wwv_flow_epg_include_mod_local(
procedure_name in varchar2)
return boolean
is
begin
-- return false; -- remove this statement when you modify this function
--
-- Administrator note: the procedure_name input parameter may be in the format:
--
-- procedure
-- schema.procedure
-- package.procedure
-- schema.package.procedure
--
-- If the expected input parameter is a procedure name only, the IN list code shown below
-- can be modified to itemize the expected procedure names. Otherwise you must parse the
-- procedure_name parameter and replace the simple code below with code that will evaluate
-- all of the cases listed above.
--
if upper(procedure_name) in (
'APEX_APPS') then
return TRUE;
else
return FALSE;
end if;
end wwv_flow_epg_include_mod_local;

everything works as designed.

Monday, December 14, 2009

My first program

Entry # 100 - celebration? Anyway, ran into probably my first program ever. Machine code, for an MC6800 (6802, actually). Dates back to 1978 or 1979:

inadd: CLRB ; clear count
CLR,X
PUSH B ; count on stack
CLR 1,X
inadd3: BSR INHEX
BMI inadd1
LDB#4
inadd2: ASL 1,X
ROL,X
DEC B
BNE inadd2
PSHS A
ORA 1,X
STA 1,X
PULS A
INC,B ; count
LDB,S
CMPB#S
BNE inadd3
inadd1: LDB,S+ ; b=count0 + #hex counts
RTS

INHEX = INCHNP

It's a counter, that should be able to count over 256, if I recall correctly. Quite complicated when you only have 8 bits. The code may not even be correct, as pencil tends to fade, and recycled paper becomes yellowish brown. Not much contrast there.
Addition: I could try it of course, as I still own a Motorola MEK6800D2 kit...