Friday, December 30, 2005

Who needs modular code?

Wouldn’t it be nice if when people wrote some useful code, they tried to make it suitably modular and reusable? This is what I have to contend with at the moment. We have a large, complex system written in Oracle Forms that we are now partially re-writing in HTMLDB (hurrah!) One function I want to replicate is the ability to change your own Oracle password; the current Forms application has a form to do this that looks like this:
Old Password: [ ] New Password: [ ] Confirm New Password: [ ]
So that should be a moment’s work to redo, right? Wrong. There is a package of procedures for user maintenance that contains the following 2 procedures that are relevant:
PROCEDURE change_password_validate
(p_username         IN VARCHAR2,
 p_old_password     IN VARCHAR2,
 p_new_password     IN VARCHAR2,
 p_confirm_password IN VARCHAR2,
 p_profile_name     IN VARCHAR2,
 p_mode             IN VARCHAR2 DEFAULT 'N',
 p_mask             IN  VARCHAR2) ;

PROCEDURE change_password_process
(p_username IN VARCHAR2,
 p_new_password IN VARCHAR2,
 p_confirm_password IN VARCHAR2,
 p_mask           IN VARCHAR2,
 p_profile_name IN VARCHAR2,
 p_encrypt_pw IN VARCHAR2,
 p_admin_mode IN VARCHAR2 DEFAULT NULL) ;
Some design flaws are immediately evident:
  • Validation is totally separate from processing. If I choose to, I can skip the validate routine altogether and call the process routine to change the password to anything I like, regardless of whether I get the old password right or confirm it correctly. (Actually, this foolish separation of validation from processing is a company standard!)
  • I get to choose whether the password is to be stored (in our own application’s USERS table) in encrypted form or not. HTF do I know whether it should be encrypted or not?
  • I need to supply something called p_mask, which I think may be something to do with the encryption process, or maybe the validation process – none of this is documented of course, or at least nobody knows where any such documentation may be found. I have tried passing the word ‘mask’ and it seems to work, except that all subsequent attempt to change the password then fail on the validation of the “old” password – perhaps because it has been encrypted in an unexpected manner.
  • I also need to supply something called p_profile_name, which I do happen to know is a user attribute something like a role, stored in the USERS table. Well excuse me, but if I’m passing in the username as a parameter, why should I have to go look up the USERS record and obtain the profile_name value just to pass it into this lazy procedure?
  • I don’t fully understand p_mode and p_admin_mode either, but at least they have defaults which I assume (for now) I can live with.
So instead of being a 5 minute job, this is probably going to occupy about a day of my time: locating source code (the packages are wrapped in the database), studying source code to see what it is doing. And if all that fails, trying to find someone from the team that wrote the code and ask them to tell me what I should be doing.

Wednesday, December 28, 2005

EAV nightmare

My charitable Christmas mood only goes so far. I'm looking through a spec which reads like a "database design nightmare!" theme advent calendar. Each page reveals a potential disaster more frightening than the one before. This is my favourite new year hang over inducing cocktail of entity attribute values and generic application design all wrapped up in a gloriously mal-specified mess.
Table: Parameters

id  VARCHAR2(50) PRIMARY KEY  --The application requesting the value
identifier NUMBER       PRIMARY KEY  --The name of the parameter
type  CHAR(1) --‘I’, ‘S’ or ‘B’ is type of value for the parameter
string_val VARCHAR2(50) --‘Y’ or ‘N’ or NULL if BOOLEAN or the string 
integer_val NUMBER       -- The integer value

note: The id holds a value comprising the IP address, type of
application and the instance at that IP.

Thursday, December 22, 2005

Bracketing frenzy

I just found myself decyphering this:

SELECT ...
FROM report_results rpt
WHERE NOT ( (substr(rpt.report_type,2,1) in ('1','2','3','4','5')) and
          (rpt.sig_type = 'ISDNMA') and
          ((rpt.group_ctn != rpt.ctn_prefix||rpt.ctn_suffix) and
           (length(rpt.group_ctn) != length(rpt.ctn_prefix||rpt.ctn_suffix)) and
           ((length(rpt.group_ctn) != (length(rpt.ctn_prefix||rpt.ctn_suffix)-1)
           ))));

Which turns out to mean this:

SELECT ...
FROM   report_results rpt
WHERE  NOT (     SUBSTR(rpt.report_type,2,1) IN ('1','2','3','4','5')
            AND  rpt.sig_type = 'ISDNMA'
            AND  rpt.group_ctn != rpt.ctn_prefix||rpt.ctn_suffix
            AND  LENGTH(rpt.group_ctn) != LENGTH(rpt.ctn_prefix||rpt.ctn_suffix)
            AND  LENGTH(rpt.group_ctn) != LENGTH(rpt.ctn_prefix||rpt.ctn_suffix)-1 );

No wonder there are no brackets left in the shops.

Friday, November 25, 2005

Code Generation Frenzy

Today's WTF moment is a bit of a monster, so I will try to explain what it does, and you can skim the actual code (this is probably a good idea) and still appreciate it.

Imagine that you have some static reference tables, let's say ORDER_STATUSES and LOCATIONS (amongst others), and you decide it would be a good idea to build an interface so that any application code can simply retrieve, say, the default order status or the head office location.

Being a bit clever, you notice that all the tables have the same structure, consisting of an ID and a description, and it occurs to you that if you UNIONed them all together, with the table name as a "type" key, you would have that generic lookup table you always wanted, along with a generic cursor to query it. (It avoids having one cursor per table, after all, so there is an efficiency gain already, right?) So you write this:

/* -------------------------------------------------------------------------------------
Name:    xxxx
Author   xxxx
Description: Script to create the types functions dynamically during schema creation
Input parameters: None
----------------------------------------------------------------------------------------*/
set serveroutput on size 1000000

variable v_destination_dir varchar2(256);
variable v_errorfilename varchar2(256);
variable v_datafile_name varchar2(256);

BEGIN
    :v_destination_dir := '&1';
    :v_errorfilename := 'type_model_functions.err';
    :v_datafile_name := 'type_model_functions.lst';
END;
/

DECLARE
    CURSOR c_tables IS 
        SELECT DISTINCT table_name
        FROM   types_functions
        ORDER BY table_name;

    CURSOR c_types IS 
        SELECT table_name, function_name, REPLACE(type_value,'''','''''') type_value 
        FROM   types_functions
        WHERE  function_name IS NOT NULL
        ORDER BY table_name,function_name;

    v_prev_rec number; 
    file_handle  UTL_FILE.FILE_TYPE :=
                 UTL_FILE.FOPEN (:v_destination_dir,  :v_datafile_name, 'W'); 

BEGIN
    UTL_FILE.PUT_LINE(file_handle, 'set define off');
    UTL_FILE.PUT_LINE
    (file_handle, '/*----------------------------------------------------------------');
    UTL_FILE.PUT_LINE
    (file_handle, ' Name: type_model_functions.lst');
    UTL_FILE.PUT_LINE
    (file_handle, ' Description: Functions to return the id from the types tables');
    UTL_FILE.PUT_LINE
    (file_handle, ' (Package Specifications)');
    UTL_FILE.PUT_LINE
    (file_handle, ' Input parameters: None');
    UTL_FILE.PUT_LINE
    (file_handle, ' Author: xxxx');
    UTL_FILE.PUT_LINE
    (file_handle, chr(10));
    UTL_FILE.PUT_LINE
    (file_handle, '-----------------------------------------------------------------*/');
    UTL_FILE.PUT_LINE(file_handle, chr(9)|| 'CREATE OR REPLACE VIEW types_view AS ');
    v_prev_rec := 0;

    FOR r_tables IN c_tables LOOP
        IF v_prev_rec <> 0 THEN
            UTL_FILE.PUT_LINE(file_handle, chr(9)|| 'UNION');
        END IF;

        UTL_FILE.PUT_LINE
        ( file_handle
        , chr(9)|| 'SELECT typeid, typelabel, ''' ||
          r_tables.table_name||''' tablename from '||r_tables.table_name );

        v_prev_rec := c_tables%rowcount;
    END LOOP;

    UTL_FILE.PUT_LINE(file_handle, '/');
    UTL_FILE.PUT_LINE(file_handle, 'show err');
    UTL_FILE.PUT_LINE(file_handle, 'drop public synonym types_view ;');
    UTL_FILE.PUT_LINE(file_handle, 'create public synonym types_view for types_view ;');
    UTL_FILE.PUT_LINE(file_handle, 'grant select on types_view to public;');

    UTL_FILE.PUT_LINE(file_handle, 'CREATE OR REPLACE PACKAGE types AS ');

    FOR r_types IN c_types LOOP
        UTL_FILE.PUT_LINE
        ( file_handle
        , chr(9) ||
          '-- Function to return the typeid for ' ||
          r_types.type_value || ' ' || r_types.table_name );

        UTL_FILE.PUT_LINE
        (file_handle, chr(9)|| 'FUNCTION '||r_types.function_name||' RETURN NUMBER;');
        UTL_FILE.PUT_LINE
        (file_handle, chr(9)|| 'PRAGMA RESTRICT_REFERENCES ('||r_types.function_name||', WNDS);');
    END LOOP;

    UTL_FILE.PUT_LINE(file_handle, 'END types;');
    UTL_FILE.PUT_LINE(file_handle, '/');
    UTL_FILE.PUT_LINE(file_handle, 'show err');
    UTL_FILE.PUT_LINE(file_handle, chr(10));

    UTL_FILE.PUT_LINE
    (file_handle, 'CREATE OR REPLACE PACKAGE BODY types AS ');
    UTL_FILE.PUT_LINE
    (file_handle, chr(9)||'CURSOR c_types_view(p_typelabel varchar2, p_tablename varchar2) IS ');
    UTL_FILE.PUT_LINE
    (file_handle, chr(9)||'SELECT typeid from types_view ');
    UTL_FILE.PUT_LINE
    (file_handle, chr(9)||'WHERE  typelabel = p_typelabel and tablename = p_tablename ;');

    FOR r_types IN c_types LOOP
        UTL_FILE.PUT_LINE
        (file_handle, chr(9)|| 'FUNCTION '|| r_types.function_name||' RETURN NUMBER IS ');
        UTL_FILE.PUT_LINE
        (file_handle, chr(9)|| chr(9)|| 'v_typeid'||chr(9)||'NUMBER; ');
        UTL_FILE.PUT_LINE
        (file_handle, chr(9)|| 'BEGIN');
        UTL_FILE.PUT_LINE
        (file_handle, chr(9)|| chr(9)|| 'OPEN  c_types_view (''' ||
                                        r_types.type_value||''','''||r_types.table_name||''');');
        UTL_FILE.PUT_LINE
        (file_handle, chr(9)|| chr(9)|| 'FETCH c_types_view INTO v_typeid;');
        UTL_FILE.PUT_LINE
        (file_handle, chr(9)|| chr(9)|| 'IF c_types_view%NOTFOUND THEN ');
        UTL_FILE.PUT_LINE
        (file_handle, chr(9)|| chr(9)|| chr(9)|| 'RAISE no_data_found; ');
        UTL_FILE.PUT_LINE
        (file_handle, chr(9)|| chr(9)|| 'END IF; ');
        UTL_FILE.PUT_LINE
        (file_handle, chr(9)|| chr(9)|| 'CLOSE  c_types_view;');
        UTL_FILE.PUT_LINE
        (file_handle, chr(9)|| chr(9)|| 'RETURN v_typeid; ');
        UTL_FILE.PUT_LINE
        (file_handle, chr(9)|| 'EXCEPTION');
        UTL_FILE.PUT_LINE
        (file_handle, chr(9)|| chr(9)|| 'WHEN OTHERS THEN');

        UTL_FILE.PUT_LINE
        ( file_handle
        , chr(9)|| chr(9)|| chr(9) ||
          'RAISE_APPLICATION_ERROR(-20001, ''Function types.' ||
          r_types.function_name || ': '''||'||sqlerrm );' );

        UTL_FILE.PUT_LINE(file_handle, chr(9)|| 'END '|| r_types.function_name|| ';');
    END LOOP;

    UTL_FILE.PUT_LINE(file_handle, 'END types;');
    UTL_FILE.PUT_LINE(file_handle, '/');

    UTL_FILE.PUT_LINE(file_handle, 'show err');

    UTL_FILE.PUT_LINE(file_handle, 'drop public synonym types;');
    UTL_FILE.PUT_LINE(file_handle, 'create public synonym types for types;');
    UTL_FILE.PUT_LINE(file_handle, 'grant execute on types to public;');

    UTL_FILE.PUT_LINE(file_handle, 'exit');

    UTL_FILE.FCLOSE (file_handle);
END;
/

@&1/type_model_functions.lst

exit
Now, given a set of values in TYPES_VALUES like this:
TABLE_NAME      FUNCTION_NAME                TYPE_VALUE
--------------- ---------------------------- ---------------
order_statuses  get_default_order_status     Default
order_statuses  get_order_cancelled_status   Cancelled
order_statuses  get_order_dispatched_status  Dispatched
order_statuses  get_order_closed             Closed
locations       get_head_office_location     Head Office
locations       get_accounts_office_location Accounts
the script generates something like this:
set define off
/*----------------------------------------------------------------------------
Name: type_model_functions.lst
Description: Packaged functions to return the typeid from the types tables
Input parameters: None
Author: xxxx
-----------------------------------------------------------------------------*/

CREATE OR REPLACE VIEW types_view AS 
SELECT typeid, typelabel, 'locations' tablename from locations
UNION
SELECT typeid, typelabel, 'order_statuses' tablename from order_statuses
/
show err
drop public synonym types_view ;
create public synonym types_view for types_view ;
grant select on types_view to public;

CREATE OR REPLACE PACKAGE types AS 
    -- Function to return the typeid for Accounts locations
    FUNCTION get_accounts_office_location RETURN NUMBER;
    PRAGMA RESTRICT_REFERENCES (get_accounts_office_location, WNDS);

    -- Function to return the typeid for Head Office locations
    FUNCTION get_head_office_location RETURN NUMBER;
    PRAGMA RESTRICT_REFERENCES (get_head_office_location, WNDS);

    -- Function to return the typeid for Default order_statuses
    FUNCTION get_default_order_status RETURN NUMBER;
    PRAGMA RESTRICT_REFERENCES (get_default_order_status, WNDS);

    -- Function to return the typeid for Cancelled order_statuses
    FUNCTION get_order_cancelled_status RETURN NUMBER;
    PRAGMA RESTRICT_REFERENCES (get_order_cancelled_status, WNDS);

    -- Function to return the typeid for Closed order_statuses
    FUNCTION get_order_closed RETURN NUMBER;
    PRAGMA RESTRICT_REFERENCES (get_order_closed, WNDS);

    -- Function to return the typeid for Dispatched order_statuses
    FUNCTION get_order_dispatched_status RETURN NUMBER;
    PRAGMA RESTRICT_REFERENCES (get_order_dispatched_status, WNDS);
END types;
/
show err


CREATE OR REPLACE PACKAGE BODY types AS 
    CURSOR c_types_view(p_typelabel varchar2, p_tablename varchar2) IS 
    SELECT typeid from types_view 
    WHERE  typelabel = p_typelabel and tablename = p_tablename ;

    FUNCTION get_accounts_office_location RETURN NUMBER IS 
        v_typeid NUMBER; 
    BEGIN
        OPEN  c_types_view ('Accounts','locations');
        FETCH c_types_view INTO v_typeid;
        IF c_types_view%NOTFOUND THEN 
            RAISE no_data_found; 
        END IF; 
        CLOSE  c_types_view;
        RETURN v_typeid; 
    EXCEPTION
        WHEN OTHERS THEN
            RAISE_APPLICATION_ERROR
            ( -20001, 'Function types.get_accounts_office_location: '||sqlerrm );
    END get_accounts_office_location;

    FUNCTION get_head_office_location RETURN NUMBER IS 
        v_typeid NUMBER; 
    BEGIN
        OPEN  c_types_view ('Head Office','locations');
        FETCH c_types_view INTO v_typeid;
        IF c_types_view%NOTFOUND THEN 
            RAISE no_data_found; 
        END IF; 
        CLOSE  c_types_view;
        RETURN v_typeid; 
    EXCEPTION
        WHEN OTHERS THEN
            RAISE_APPLICATION_ERROR
            ( -20001, 'Function types.get_head_office_location: '||sqlerrm );
    END get_head_office_location;

    FUNCTION get_default_order_status RETURN NUMBER IS 
        v_typeid NUMBER; 
    BEGIN
        OPEN  c_types_view ('Default','order_statuses');
        FETCH c_types_view INTO v_typeid;
        IF c_types_view%NOTFOUND THEN 
            RAISE no_data_found; 
        END IF; 
        CLOSE  c_types_view;
        RETURN v_typeid; 
    EXCEPTION
        WHEN OTHERS THEN
            RAISE_APPLICATION_ERROR
            ( -20001, 'Function types.get_default_order_status: '||sqlerrm );
    END get_default_order_status;

    FUNCTION get_order_cancelled_status RETURN NUMBER IS 
        v_typeid NUMBER; 
    BEGIN
        OPEN  c_types_view ('Cancelled','order_statuses');
        FETCH c_types_view INTO v_typeid;
        IF c_types_view%NOTFOUND THEN 
            RAISE no_data_found; 
        END IF; 
        CLOSE  c_types_view;
        RETURN v_typeid; 
    EXCEPTION
        WHEN OTHERS THEN
            RAISE_APPLICATION_ERROR
            ( -20001, 'Function types.get_order_cancelled_status: '||sqlerrm );
    END get_order_cancelled_status;

    FUNCTION get_order_closed RETURN NUMBER IS 
        v_typeid NUMBER; 
    BEGIN
        OPEN  c_types_view ('Closed','order_statuses');
        FETCH c_types_view INTO v_typeid;
        IF c_types_view%NOTFOUND THEN 
            RAISE no_data_found; 
        END IF; 
        CLOSE  c_types_view;
        RETURN v_typeid; 
    EXCEPTION
        WHEN OTHERS THEN
            RAISE_APPLICATION_ERROR
            ( -20001, 'Function types.get_order_closed: '||sqlerrm );
    END get_order_closed;

    FUNCTION get_order_dispatched_status RETURN NUMBER IS 
        v_typeid NUMBER; 
    BEGIN
        OPEN  c_types_view ('Dispatched','order_statuses');
        FETCH c_types_view INTO v_typeid;
        IF c_types_view%NOTFOUND THEN 
            RAISE no_data_found; 
        END IF; 
        CLOSE  c_types_view;
        RETURN v_typeid; 
    EXCEPTION
        WHEN OTHERS THEN
            RAISE_APPLICATION_ERROR
            ( -20001, 'Function types.get_order_dispatched_status: '||sqlerrm );
    END get_order_dispatched_status;
END types;
/
show err
drop public synonym types;
create public synonym types for types;
grant execute on types to public;
exit

Now all you have to do is ensure that there are values in the LOCATIONS table with typelabels 'Head Office', 'Accounts' and so on, and whenever you run the script it will create or replace a whole new package implementing whatever you put in TYPES_FUNCTIONS, and application code can simply invoke types.get_accounts_office_location().

Now, I wonder whether there is a way to generate the TYPES_FUNCTIONS table...

Friday, November 11, 2005

When RTFM Goes Wrong IV

The sheer size of Oracle's documentation set always astonishes me, and it's a wonder that any of it ever manages to evolve across versions. But sure enough, the important bits usually get a brush-up during major release changes and a few new bits get thrown in. But not so for a particularly dry part of Chapter 2 of the "Application Developers Guide - Fundamentals" for 10g Release 2 (Chapter 5 in earlier versions), subtitled "Improving Transaction Performance".

Remember this is 10.2 - it's bang up to date as far as Oracle releases go. I don't think that any further comment is required here else the post will get too long so I'll just send you to the link and suggest you take particular note of the comments on stored PL/SQL functions, ANALYZE and explicit cursors.

http://download-uk.oracle.com/docs/cd/B19306_01/appdev.102/b14251/adfns_sqlproc.htm#sthref179

The rollback segment entry can be forgiven as not everyone is using UNDO yet (though I always hated it as a piece of advice when I used rollback segments).

You can never have too much redirection

And who needs PL/SQL anyway when you already have all the procedural control you need, thanks to Mr Stephen Bourne and his excellent Shell?

echo -------------------------------------  >> $LOGS/$MY_LOGFILE_NAME
echo Load the xxx schema tables for xyz >> $LOGS/$MY_LOGFILE_NAME
echo ------------------------------------- >> $LOGS/$MY_LOGFILE_NAME
sqlplus -s $XXX_USERNAME/$XXX_PASSWORD @$XYZ_BIN/xyz_load_xxx_data.sql $LOGS $MY_ERRORFILE_NAME \
>> $LOGS/$MY_LOGFILE_NAME retcode=`echo \$?` if [ "$retcode" -ne 0 ] ;then echo Step failed >> $LOGS/$MY_LOGFILE_NAME exit $RETURN_GENERIC_FAIL fi echo ------------------------------------ >> $LOGS/$MY_LOGFILE_NAME echo Processing abc files >> $LOGS/$MY_LOGFILE_NAME echo ------------------------------------ >> $LOGS/$MY_LOGFILE_NAME $xyz_BIN/xyz_process_abc_ndt_files.pl retcode=`echo \$?` if [ "$retcode" -ne 0 ] ;then echo Step failed >> $LOGS/$MY_LOGFILE_NAME exit $RETURN_GENERIC_FAIL fi echo ------------------------------------ >> $LOGS/$MY_LOGFILE_NAME echo Processing MF files >> $LOGS/$MY_LOGFILE_NAME echo ------------------------------------ >> $LOGS/$MY_LOGFILE_NAME $xyz_BIN/xyz_process_def_ndt_files.pl retcode=`echo \$?` if [ "$retcode" -ne 0 ] ;then echo Step failed >> $LOGS/$MY_LOGFILE_NAME exit $RETURN_GENERIC_FAIL fi echo ------------------------------------- >> $LOGS/$MY_LOGFILE_NAME echo Merge MF and customer tables for xyz >> $LOGS/$MY_LOGFILE_NAME echo ------------------------------------- >> $LOGS/$MY_LOGFILE_NAME sqlplus -s $XYZ_USERNAME/$XYZ_PASSWORD \
@$XYZ_BIN/xyz_merge_mf_and_customer_tables.sql $LOGS $MY_ERRORFILE_NAME \ >> $LOGS/$MY_LOGFILE_NAME retcode=`echo \$?` if [ "$retcode" -ne 0 ] ;then echo Step failed >> $LOGS/$MY_LOGFILE_NAME exit $RETURN_GENERIC_FAIL fi echo ------------------------------------- >> $LOGS/$MY_LOGFILE_NAME echo Populate equipment table for xyz >> $LOGS/$MY_LOGFILE_NAME echo ------------------------------------- >> $LOGS/$MY_LOGFILE_NAME sqlplus -s $XYZ_USERNAME/$XYZ_PASSWORD \
@$XYZ_BIN/xyz_create_equipment.sql $LOGS $MY_ERRORFILE_NAME \
>> $LOGS/$MY_LOGFILE_NAME retcode=`echo \$?` if [ "$retcode" -ne 0 ] ;then echo Step failed >> $LOGS/$MY_LOGFILE_NAME exit $RETURN_GENERIC_FAIL fi echo ------------------------------------- >> $LOGS/$MY_LOGFILE_NAME echo Populate locations table for xyz >> $LOGS/$MY_LOGFILE_NAME echo ------------------------------------- >> $LOGS/$MY_LOGFILE_NAME sqlplus -s $XYZ_USERNAME/$XYZ_PASSWORD \
@$XYZ_BIN/xyz_create_locations.sql $LOGS $MY_ERRORFILE_NAME \
>> $LOGS/$MY_LOGFILE_NAME retcode=`echo \$?` if [ "$retcode" -ne 0 ] ;then echo Step failed >> $LOGS/$MY_LOGFILE_NAME exit $RETURN_GENERIC_FAIL fi runendt=`date` echo ---------------------------------------------------- >> $LOGS/$MY_LOGFILE_NAME echo "xyz dataload Ended - $runendt ....." >> $LOGS/$MY_LOGFILE_NAME echo ---------------------------------------------------- >> $LOGS/$MY_LOGFILE_NAME echo >> $LOGS/$MY_LOGFILE_NAME exit $RETURN_SUCCESS

Friday, October 14, 2005

Spot the difference

Friday, October 07, 2005

Prog rock

Anyone who remembers The Ramones will have happy memories of their toetappingly monolithic slabs of pure punk pop. After a couple of albums of two-chord magic, however, there were those who said it was starting to sound samey, and I believe it was Joey Ramone's dad who commented in an interview that perhaps the boys might want to "complicate it up a bit".

It seems the authors of Oracle Database 10g PL/SQL Programming (Oracle Press, 2004) chose to follow similar advice in their chapter on working with 8i, er, I mean 10g collections. Anyone looking for a funkier way to DELETE FROM addresses WHERE address_id = 11, for example, need look no further:
www.peakretrieval.com/plsql/Chapter6/varray_dml1.sql

And while we're at it,

SELECT column_value
FROM   THE (SELECT CAST(emp_address AS varray_nested_table )
            FROM   emp
            WHERE  empno = 11);

(love that type name btw) could also be written as

SELECT column_value
FROM   emp, TABLE(emp_address)
WHERE  empno = 11;

but where's the fun in that?

Tuesday, October 04, 2005

You can never have too many tables

There is no problem that cannot be solved with a loop and some dynamic SQL.

Problem: Your report can return rows for more than one feature code. How on Earth are you going to store the results?

Solution: Loop through the FEATURES table, creating three dynamically named tables for each feature code. Problem solved.

BEGIN
FOR rec IN
    ( SELECT f.short_code ft FROM features f
      WHERE  f.chargeable = 'Y' )
LOOP
    BEGIN
        EXECUTE IMMEDIATE 'drop table r_t8_' || rec.ft ;
    EXCEPTION
        WHEN OTHERS THEN NULL;
    END;

    EXECUTE IMMEDIATE 'create table r_t8_' || rec.ft || '(' ||
    'report_type varchar2(50),' ||
    'report_date date,' ||
    'switch_id number,' ||
    'feat_count number,' ||
    'rental_rate number,' ||
    'discount number,' ||
    'rental_rating number)' ;

    BEGIN
        EXECUTE IMMEDIATE 'drop table r_t7_' || rec.ft ;
    EXCEPTION
        WHEN OTHERS THEN NULL;
    END;

    EXECUTE IMMEDIATE 'create table r_t7_' || rec.ft || '(' ||
    'report_type varchar2(50),' ||
    'report_date date,' ||
    'switch_id number,' ||
    'feat_count number,' ||
    'rental_rate number,' ||
    'discount number,' ||
    'rental_rating number)' ;

    BEGIN
        EXECUTE IMMEDIATE 'drop table r_t9_' || rec.ft ;
    EXCEPTION WHEN OTHERS THEN
        NULL;
    END;

    IF rec.ft <> 'ZZ' THEN
        EXECUTE IMMEDIATE 'create table r_t9_' || rec.ft || '(' ||
        'report_type varchar2(50),' ||
        'report_date date,' ||
        'switch_id number,' ||
        'feat_count number,' ||
        'rental_rate number,' ||
        'discount number,' ||
        'rental_rating number)' ;
    END IF;
END LOOP;
END;
/

Now, how to write those reports so they populate the right tables? I know...

Monday, October 03, 2005

Watching the Watcher-Watchers

I've just found a process that uses on-update triggers to insert rows into a table:
Queue_Monitor_Table
( source_table_name varchar2(30)
  rowid             varchar2(16) /*lovely,  but not the point*/
  enqueued_yn       char(1)
  dequeued_yn       char(1) )
This is processed by a pl/sql "queue_monitor_listener" that is started in a background process. A "queue_monitor_listener" if you can't guess, repeatedly scans the table for rows where enqueued_yn = 'N' and then it enqueues the rowid to an AQ queue, called the "row_update_queue". When it's happy that a row has been enqueued, it updates the enqueued_yn flag to "Y". As you might fear, there is also dequeue process. This also runs in a background job. When this has successfully dequeued from the row_update_queue, and done its task ( which could be the subject of a new WTF ), it updates the Queue_Monitor_Table to set dequeued_yn = 'Y'. Nice. The fun really seems to have begun when, in order to improve performance, it was decided that several "queue_monitor_listener" processes should be started in parallel background processes.

Thursday, September 29, 2005

Never do in SQL what you can do in PL/SQL

Back when I first started out as a trainee Forms developer, one of my colleagues had the bright idea of implementing some "CONNECT BY PRIOR" logic step by step, in Forms. The trigger would add a new blank record to the block, then retrieve the next row into it with SELECT INTO, then loop around until there were no more rows to process or Hell froze over. Perhaps unsurprisingly, it was a close thing.

I mentioned to Scott Swank that we were always on the lookout for material, and he sent me the following code that he'd come across (after editing it for brevity and anonymity), and somehow it reminded me of that old Forms trigger. This pattern is repeated ten times throughout the code, checking for various child attributes by (how else?) looping through all the records one by one until the desired attribute is found not to be true (and if something is not untrue then it must be true, right?)

PROCEDURE has_an_attribute
    ( x IN VARCHAR2, resultout OUT VARCHAR2 )
IS
    l_parent_id   VARCHAR2 (30);

    CURSOR c_child
    IS
        SELECT an_attribute
        FROM   child_table
        WHERE  parent_id = l_parent_id;
BEGIN
    l_parent_id := some_function(x);
    resultout := 'COMPLETE:Y';

    FOR v_child IN c_child
    LOOP
        IF v_child.an_attribute = 'N'
        THEN
            resultout := 'COMPLETE:N';
            EXIT;
        END IF;
    END LOOP;
END;

Wednesday, September 28, 2005

Check List

( ) Written dodgy concatenated embedded sql?
( ) Wrapped it in shell script?
( ) Littered it with literals?
( ) Included some random control characters?
( ) Released it without checking it works?

( ) Remembered to add the sarcastic comment questioning someone else's ability?
# formatting had to be included in select statement
# could this suggest the database could use some normalisation?
strqry="select invoice_line_items.product_code||"
strqry=$strqry"invoice_line_items.customer_type||"
strqry=$strqry"invoice_line_items.product_sku||\n^D"
strqry=$strqry"substr(expiry_date,3,4)||"
strqry=$strqry"to_char(trunc(instant_cost/10000),'FMX')||"
strqry=$strqry"to_char(mod(instant_cost,10000),'FM0999')||"
strqry=$strqry"contract_type"
strqry=$strqry"from invoice_line_items, retail_calendar, customer_contract,"
strqry=$strqry"product_domain"
strqry=$strqry"where last_retail_day >= retail_calendar.retail_day"
strqry=$strqry"and first_retail_day <= retail_calendar.retail_day"
strqry=$strqry"and retail_calendar.day = $day"
strqry=$strqry"and retail_calendar.month = $month"
strqry=$strqry"and retail_calendar.year = $year"
strqry=$strqry"and invoice_line_items.line_code='$flag'"
strqry=$strqry"and customer_contract.line_code='$flag'"
strqry=$strqry"and invoice_line_items.product_code=customer_contract.product_code"
strqry=$strqry"and invoice_line_items.customer_type="
strqry=$strqry"      customer_contract.customer_type"
strqry=$strqry"and invoice_line_items.product_sku="
strqry=$strqry"      customer_contract.product_sku"
if [ $host_id != "-1" ]; then
        strqry=$strqry" and product_domain.host_id = $host_id"
fi
strqry=$strqry"and invoice_line_items.product_code=product_domain.product_code"
strqry=$strqry"and invoice_line_items.customer_type="
strqry=$strqry"      product_domain.customer_type"
strqry=$strqry"and invoice_line_items.product_sku="
strqry=$strqry"      product_domain.product_sku;"

Thursday, September 22, 2005

Fun with String

It seems that my predecessor needed to convert a row of five columns into a column with five rows. Simple enough, you might think, once you realise that you can use a handy collection type in a TABLE() expression. But hey, why stop there when there is a much, much harder way involving a string-to-table function and a self-join on three columns? ...that aren't quite unique... (Table and column names changed to protect the guilty)
SELECT ...
FROM   huge_table t
     , TABLE ( SELECT utils.string_to_table
                      ( t2.col1 || ',' || t2.col2 || ',' ||
                        t2.col3 || ',' || t2.col4 || ',' || t2.col5)
               FROM   huge_table t2
               WHERE  t2.switch_id = t.switch_id
               AND    t2.switch_ctn = t.switch_ctn
               AND    t2.phone_prefix = t.phone_prefix
               AND    ROWNUM < 2 )
Now if that isn't half ample in the WHERE, I don't know what is.

Thursday, September 15, 2005

Joins explained

Remember to print out the following definitions and keep them by your desk in case you forget what a complex right inner join is (from the article "Database design for platform independence", TechRepublic 2002):

Don't use joins
Joins become a problem with Oracle and SQL Server DBMSs because the two systems have fundamentally different approaches to the concept. Basically, joins don’t always work the same on every DBMS, and you can end up with unexpected result sets. Before we talk about the differences and how to get around them, you should understand the basic types of join clauses:
  • Join
    Joins are powerful SQL commands for creating tables from data that is retrieved from multiple sources.
  • Equi-Join
    Equi-Joins are formed by retrieving all the data from two separate sources and combining it into one, large table.
  • Inner/Outer Join
    Inner Joins are joined on the inner columns of two tables. Outer Joins are joined on the outer columns of two tables.
  • Left/Right Join
    Left Joins are joined on the left columns of two tables. Right Joins are joined on the right columns of two tables.
  • Compound/Complex Joins
    There are also other kinds of joins—left/inner, left/outer, right/inner, and right/outer.
The important thing to remember about joins is to not use them if you want your application to work well with different database servers.

I love the way that it's exactly the kind of thing you would make up, if you were 12 and had absolutely no idea how to answer a homework question.

While I'm at it I can't help quoting a bit more, this time about the well-known ADD command:

ANSI SQL: CREATE, DROP, ADD, UPDATE, DELETE, INSERT, SELECT
As a rule, the only commands you should use are:

  • CREATE and DROP for Data Definition Language (DDL).
  • ADD, UPDATE, DELETE and INSERT for Data Manipulation Language (DML).
  • SELECT for data retrieval.

Friday, September 09, 2005

Twirled Half Ample in the WHERE

It is of course completely unfair to post this message by someone who clearly does not speak English. Actually you have to admire his valiant effort to describe a problem with dynamic ref cursors in a foreign language, through what must be the world's most eccentric translation software, or possibly using only a rather old dictionary. (Credit also goes to Art Metzer who actually provided a solution.) But it remains our all-time favourite Oracle forum post, so here it is:
Hi!!

I am with one doubts, I mounted one procedure that to need to return a cursor.

The STRING SQL is must be twirled half ample in the "WHERE".

For that haven't to make varios ifs, would like to know if is possible to create one parameter that it comes with where and to place in the routine to twirl, I tried to make this but not gave certain.

It below follows:

OPEN RCURSOR FOR SELECT * FROM TABELA1 WHERE CAMPO1 < 10 || VARIAVEL1

In var(VARIAVEL1) it would be:

'AND CAMPO2 = 3 AND CAMPO3 > = 25'

For example, this would be possible?

Or I will have exactly that to mount one 'IF' giant for to treat all the routine.

Tks.

See www.orafaq.com/forum/mv/msg/4467/12407/0 for the original post.

Saturday, September 03, 2005

If at first you don't succeed

Just came across this in a rather large query I have to fix:
NVL(di_core.di_tools.area_code_translate(v_switch_id,d.dncode),
    di_core.di_tools.area_code_translate(v_switch_id,d.dncode))

Friday, September 02, 2005

Mail it

I recently received the following request on a web forum:
Hello,

Please let me know how can we send an oracle database via e-mail to other people. Please reply ASAP as it is very urgent. The project is struck because of this.

Thanks in advance
Hmm. I replied asking for some descripton of the business requirement, and mentioned exp/imp utilities, .dmp files being typically smaller than an entire database, possibly XML, etc. Back came the reply:
Thank you sir for your inputs.

Actually I am from core electronics, VLSI background.

We need to configure a database (it contains all the information of components that can be used).

I have worked with Access database(which we can send and receive by zipping and sending).

Now the problem is that the person1 who is else where in globe has to send me the oracle database. Now we dont know how do we do that.We dont even have those passwords or anything... I just have no clue wht it is like .. wht options.. nothing

Now could you please tell me how can we send the database

Thanks
Oh dear. What is the file extension of the files they are sending you? I asked. Do you have an Oracle license? I must admit I had to look up VLSI ("Very Large Systems Integration", of course). And then when all hope seemed lost:
got my way

I asked the person having database to put it on FTP...

i would then be able to get that whole folder and hence the database...

Thanks for your time
Problem solved I think.

Wednesday, August 31, 2005

I'm not making it up

This gem is used at times of particularly high throughput to monitor the size of queues.
PROCEDURE getqueuecount (
   queuename                  IN       VARCHAR2
  ,agentname                  IN       VARCHAR2
  ,queuecount                 OUT      INTEGER
)
IS
   queueoptions        DBMS_AQ.dequeue_options_t;
   messageproperties   DBMS_AQ.message_properties_t;
   MESSAGE             connect_db_external_q_type;
   messageid           RAW (16);
   nodequeuewaiting    EXCEPTION;
   dequeuewaiting      BOOLEAN                      := TRUE;
   nolock              EXCEPTION;
   PRAGMA EXCEPTION_INIT (nolock, -054);
   PRAGMA EXCEPTION_INIT (nodequeuewaiting, -25228);
BEGIN
   LOCK TABLE q_lock_table IN EXCLUSIVE MODE NOWAIT;
   queuecount                 := 0;
   queueoptions.WAIT          := 1;
   queueoptions.consumer_name := agentname;
   queueoptions.navigation    := DBMS_AQ.first_message;

   BEGIN
      WHILE (dequeuewaiting)
      LOOP
         DBMS_AQ.dequeue (queuename
                         ,queueoptions
                         ,messageproperties
                         ,MESSAGE
                         ,messageid
                         );
         queuecount  :=   queuecount  + 1;
      END LOOP;
   EXCEPTION
      WHEN nodequeuewaiting
      THEN
         dequeuewaiting             := FALSE;
   END;

   ROLLBACK;
EXCEPTION
   WHEN nolock
   THEN
      queuecount                 := -1;
END;

Tuesday, August 30, 2005

Nesting instinct

It is said that the human brain can retain only seven facts in short term memory at one time. Or something like that. Test yourself anyway with the following example, which occurs in the middle of a 175-line INSERT statement, in a 1200-line package body, all written in much the same self-explanatory style:
INSERT INTO some_unsuspecting_table
WITH all_dn AS (SELECT ... FROM ...)
SELECT ...
       CASE WHEN SUBSTR(all_dn.dn_data,6,4) IN ('PODN','UNDN')
           THEN SUBSTR(all_dn.dn_data,6,4)
           ELSE
               NVL2
               ( NVL(l.len, substr(a.features,16,15)),
                 DECODE(MAX(CASE WHEN all_dn.dn_data LIKE '%MSN%'
                                 THEN 2
                                 ELSE
                                     CASE WHEN all_dn.dn_data LIKE '%TWDN%'
                                          THEN 1
                                     END
                            END)
                        OVER (PARTITION BY NVL(l.len, SUBSTR(a.features,16,15) ) ),
                        2, 'MSN',
                        1, 'TWDN', 'LINE' ),
                 CASE WHEN all_dn.dn_data LIKE '%MSN%'
                      THEN 'MSN'
                      ELSE all_dn.dn_data
                 END )
       END main_service_type
     , ...and so on...
FROM   etc etc;
Now, how many levels of nesting was that? What, you passed out somewhere around OVER (PARTITION BY NVL(l.len, SUBSTR(a.features,16,15) ) )? Shame on you.

The little things that snowball

There's nothing more I can say about this.
CREATE TABLE INVOICE_LINEITEM 
(
...
...
MONTH_OF_YEAR   CHAR(1)
...
)

Avoiding the overhead of SQL

Ok, I'll go first, even though this is more of a "whyTF". Have a look at this piece of genius. I've only pulled the bare bones because these scripts are HUGE. Names are changed for the usual reasons. dumpdb.sh follows:
#!/bin/ksh

function showtables
{
  cat << EOF
acc:client_account
amt:client_model_type
...
700 lines later
...
ugp:ultimate_group_parameter
vap:validation_parameter
EOF
}

function execSQL
{
eval "echo \"$SQL_HEADER$1$SQLTRAILER\" \
  |sqlplus -S $ORACONNECT   |path=/usr/xpg4/bin:"$PATH" awk -F: $AWK_PROG \
  | sed 's/::/: :/g' | sed 's/~:/:/g' \
  $OUT_FILTER \
  $ OUTPUT"
}
.
. 700 functions like this, one per table.
.
function fDump_client
{
    execSQL "select * from client;"
}
.

for table in $tables_to_dump
do
    if [[ "$TO_TAB_FILE" == "Y" ]]; then
        ## Send output to file named <table>.csv
        export OUTPUT=" > $table.csv"
        echo "Dumping $table to $table.csv"
    fi
    # Get filter if required

    if [[ $? -eq 0 ]]; then
        # There is a specific function for this table so use it
        fDump_${table}

    else
        # No function - just use default SELECT
        execSQL "select * from ${table};"
    fi
done

which is called by "application support scripts" as demonstrated:
#!/bin/ksh

function select_client_access_type
{
    
    # dbselect gives:
    #
    # 1 CLIENT_KEY         NOT NULL CHAR(3)
    # 2 MEMBER_KEY         VARCHAR2(4)
    # 3 NAME               VARCHAR2(50)
    # 4 PHONE              VARCHAR2(22)
    # 5 ACCESS_ENABLED     VARCHAR2(1)
    # 6 ACCESS_TYPE        CHAR(1)
    # 7 MM_AND_OR_CLIENT   VARCHAR2(1)
    # 8 RIGHTS_GROUP_ID    NUMBER(4)
    # 9 MANAGER_ID         NUMBER(38)

    IFS=:
    if ! set -- $(dumpdb | grep "^${CLIENT_KEY}:")
    then
        echo "CLIENT '$CLIENT_KEY' not found"
        exit $EXIT_NO_CLIENT
    fi

    MEMBER_KEY="$2"
    ACCESS_ENABLED="$5"
    ACCESS_TYPE="$6"
    MM_AND_OR_CLIENT="$7"
    RIGHTS_GROUP_ID="$8"

    if [[ $ACCESS_ENABLED != Y ]]
    then
    if [[ $ACCESS_TYPE != T ]]
    then
        echo "CLIENT ACCESS_TYPE is $ACCESS_TYPE"
    fi
    fi
}