Wednesday, October 10, 2012

Procedure to automate datafiles' resizing taking into account filesystems' free space

I wanted to create a mechanism for our operators to be able to give more space to tablespaces, automatically, when there is a need to.
The operators don't know the datafiles' location and the filesystems' free space, so the only thing they will do is to run a schell script with the tablespace name as a parameter.

Unfortunately, oracle database does not store any information about the filesystems and their space usage, so if you are not careful you may end up with completely full filesystems.
We can only get this info, using OS commands like bdf and df, so to make it available to our database, we will create a text file in our server, containing the output of those commands and an external table in the database to read that file.

I created this example in HP-UX, so you will need to do a few modifications to make it run in Linux or other Unix.
For example, bdf is an HP-UX command and you should use df in other Unix/Linux.

First, create a directory pointing to the location, where our scripts and output files will be stored, e.g. /oradb/orabin/scripts:
CREATE DIRECTORY SCRIPT_DIR AS '/oradb/orabin/scripts';

Now, create the external table, called df, which reads /oradb/orabin/scripts/bdf.txt:
CREATE TABLE DF
                   (FILESYSTEM VARCHAR2(200),
                    KBYTES       NUMBER,
                    USED         NUMBER,
                    AVAIL        NUMBER,
                    USED_PER     VARCHAR2(20),
                    MOUNTED      VARCHAR2(100)
                   )
     ORGANIZATION EXTERNAL
     (
       TYPE ORACLE_LOADER
       DEFAULT DIRECTORY SCRIPT_DIR
       ACCESS PARAMETERS
       (
         RECORDS DELIMITED BY NEWLINE
         BADFILE SCRIPT_DIR:'bdf%a_%p.bad'
         LOGFILE SCRIPT_DIR:'bdf%a_%p.log'
         FIELDS TERMINATED BY ' '
         MISSING FIELD VALUES ARE NULL
       )
       LOCATION ('bdf.txt')
     )
     REJECT LIMIT UNLIMITED;

Our script will make some of the given tablespace's datafiles autoextensible, taking into account the filesystems' free space.
We discard any datafiles there are in filesystems with > 99% of used space.
For all others, we autoextend them to a size equal to:
CURRENT DATAFILE SIZE + 10% OF FILESYSTEM'S FREE SPACE
If the above size is greater than tablespace's block size * 4194302, then we set MAXSIZE UNLIMITED, to avoid to get an ORA error.
So, create file /oradb/orabin/scripts/extend.sql:
SET SERVEROUTPUT ON
SPOOL OUT.TXT

DECLARE
   V_STMT    VARCHAR2 (1000);
   V_BSIZE   NUMBER;

   CURSOR C1
   IS
      SELECT A.FILE_NAME,
             A.BYTES,
             B.AVAIL,
             (BYTES / 1024) + (AVAIL * 0.1) NEW_SIZE
        FROM DBA_DATA_FILES A, DF B
       WHERE     A.TABLESPACE_NAME = UPPER ('&1')
             AND SUBSTR (A.FILE_NAME, 1, INSTR (A.FILE_NAME, '/', -1, 2) - 1) = B.MOUNTED
             AND SUBSTR (B.USED_PER, 1, INSTR (B.USED_PER, '%') - 1) < 99;
BEGIN
   SELECT BLOCK_SIZE
     INTO V_BSIZE
     FROM DBA_TABLESPACES
    WHERE TABLESPACE_NAME = UPPER ('&1');

   FOR DFILE IN C1
   LOOP
      DBMS_OUTPUT.PUT_LINE ('Filename: ' || DFILE.FILE_NAME);
      DBMS_OUTPUT.PUT_LINE ('Size (KB): ' || DFILE.BYTES / 1024);
      DBMS_OUTPUT.PUT_LINE ('Avail (KB): ' || DFILE.AVAIL);

      IF DFILE.NEW_SIZE < V_BSIZE * 4194302 / 1024
      THEN
         V_STMT := 'ALTER DATABASE DATAFILE ''' || DFILE.FILE_NAME || ''' AUTOEXTEND ON NEXT 128M MAXSIZE ' || DFILE.NEW_SIZE || 'K';
      ELSE
         V_STMT := 'ALTER DATABASE DATAFILE ''' || DFILE.FILE_NAME || ''' AUTOEXTEND ON NEXT 128M MAXSIZE UNLIMITED';
      END IF;

      DBMS_OUTPUT.PUT_LINE (V_STMT);
      EXECUTE IMMEDIATE V_STMT;
      DBMS_OUTPUT.PUT_LINE ('--------------------------');
   END LOOP;
END;
/

SPOOL OFF

Finally, we create the shell script /oradb/orabin/scripts/extend.sh, which creates the bdf.txt with the appropriate format for our external table. It also sends an email to the DBAs, so they get informed that the procedure run and which datafiles were affected.
#!/bin/sh
ORACLE_HOME=/oradb/orabin/app/product/db1
export ORACLE_HOME
ORACLE_SID=OTE
export ORACLE_SID

rm bdf*
rm original.txt
bdf|grep -v File|grep -v vol|grep -v odm|grep -v shared>original.txt
sed -e "s/  */ /g" original.txt > bdf.txt

cd /oradb/orabin/scripts
sqlplus -s /nolog << EOF
connect / as sysdba
@extend.sql $1
exit
EOF

cat OUT.TXT|mailx -s "Autoextend script run for $ORACLE_SID" name@ote.gr

Now, the only thing the operator has to do is run extend.sh with the tablespace name as a parameter:
extend.sh user_data

Monday, September 24, 2012

View alert log from sql

In 11g, there is an easy way to read you database's alert log, using table X$DBGALERTEXT:
SELECT ORIGINATING_TIMESTAMP,
         MODULE_ID,
         PROCESS_ID,
         MESSAGE_TEXT
    FROM X$DBGALERTEXT
ORDER BY ORIGINATING_TIMESTAMP DESC;

In pre-11g databases, you have to read it, using another way, like UTL_FILE package.
The following procedure, will put in an ALERT.LOG file a percent [default 5%] of your database's alert log.
If you want all your alert log to be fetched, then set V_PERCENT = 1.
SET SERVEROUTPUT ON
SPOOL ALERT.LOG

DECLARE
   V_PERCENT    NUMBER := .05;
   V_BDUMP      VARCHAR2 (1000);
   V_NAME       VARCHAR2 (20);
   V_FILE       VARCHAR2 (20);
   V_LINE       VARCHAR2 (8000);
   V_HANDLE     UTL_FILE.FILE_TYPE;
   V_DIR_STMT   VARCHAR2 (100);
   V_SIZE       INTEGER;
   V_BSIZE      INTEGER;
   V_EXISTS     BOOLEAN;
BEGIN
   SELECT VALUE
     INTO V_BDUMP
     FROM V$PARAMETER
    WHERE NAME = 'background_dump_dest';

   DBMS_OUTPUT.PUT_LINE (
      '**************************** INFO *************************');
   DBMS_OUTPUT.PUT_LINE ('BDUMP: ' || V_BDUMP);

   SELECT VALUE
     INTO V_NAME
     FROM V$PARAMETER
    WHERE NAME = 'instance_name';

   DBMS_OUTPUT.PUT_LINE ('DB NAME: ' || V_NAME);

   V_FILE := 'alert_' || V_NAME || '.log';
   DBMS_OUTPUT.PUT_LINE ('FILENAME: ' || V_FILE);

   V_DIR_STMT := 'CREATE DIRECTORY ALERT AS ''' || V_BDUMP || '''';
   DBMS_OUTPUT.PUT_LINE ('COMMAND: ' || V_DIR_STMT);

   EXECUTE IMMEDIATE V_DIR_STMT;

   V_HANDLE := UTL_FILE.FOPEN ('ALERT', V_FILE, 'r');

   UTL_FILE.FGETATTR ('ALERT',
                      V_FILE,
                      V_EXISTS,
                      V_SIZE,
                      V_BSIZE);
   DBMS_OUTPUT.PUT_LINE ('ALERT SIZE: ' || V_SIZE / 1024 / 1024 || ' MB');

   UTL_FILE.FSEEK (V_HANDLE, NULL, V_SIZE * (1 - V_PERCENT));
   DBMS_OUTPUT.PUT_LINE (
      'FETCHED ALERT SIZE: ' || V_SIZE * (V_PERCENT / 1024 / 1024) || ' MB');
   DBMS_OUTPUT.PUT_LINE (
      '********************************************************');

   LOOP
      BEGIN
         UTL_FILE.GET_LINE (V_HANDLE, V_LINE);
         DBMS_OUTPUT.PUT_LINE (V_LINE);
      EXCEPTION
         WHEN NO_DATA_FOUND
         THEN
            EXIT;
      END;
   END LOOP;

   UTL_FILE.FCLOSE (V_HANDLE);

   EXECUTE IMMEDIATE 'DROP DIRECTORY ALERT';
END;
/

SPOOL OFF;

Saturday, August 4, 2012

Calculate actual free space of datafiles and tablespaces

Let's say, we have a 10GB datafile, autoexendible to 20GB. Querying DBA_FREE_SPACE we see this datafile has 2GB free space.
The actual free space is MAX_SIZE - SIZE + FREE_SPACE, so for our example is 20GB - 10GB + 2GB = 12GB.
The following query will display this datafiles' free space [TOTAL FREE SPACE GB] per tablespaces.
  SELECT B.TABLESPACE_NAME,
         B.FILE_NAME,
         ROUND (B.BYTES / 1024 / 1024 / 1024, 2) "SIZE GB",
         ROUND (B.MAXBYTES / 1024 / 1024 / 1024, 2) "MAX SIZE GB",
         ROUND (SUM (A.BYTES) / 1024 / 1024 / 1024, 2) "FREE SPACE GB",
         CASE
            WHEN MAXBYTES = 0
            THEN
               ROUND (SUM (A.BYTES) / 1024 / 1024 / 1024, 2)
            ELSE
               ROUND (
                    ( (B.MAXBYTES - B.BYTES) + SUM (A.BYTES)) / 1024 / 1024 / 1024, 2)
         END
            "TOTAL FREE SPACE GB"
    FROM DBA_FREE_SPACE A, DBA_DATA_FILES B
   WHERE A.FILE_ID = B.FILE_ID AND B.TABLESPACE_NAME IN ('CBAM')
GROUP BY B.TABLESPACE_NAME,
         B.FILE_NAME,
         B.BYTES,
         B.MAXBYTES
ORDER BY 1, 2;

The output is:   
TABLESPACE_NAMEFILE_NAMESIZE GBMAX SIZE GBFREE SPACE GBTOTAL FREE SPACE GB
CBAM/oradb/SOADB/oradata1/cbam.dbf9.77000
CBAM/oradb/SOADB/oradata6/cbam14.dbf19.5319.5300
CBAM/oradb/SOADB/oradata6/cbam18.dbf19.5319.530.010.01
..................
CBAM/oradb/SOADB/oradata6/cbam27.dbf12.52511.9224.42
CBAM/oradb/SOADB/oradata7/cbam25.dbf25250.030.03
CBAM/oradb/SOADB/oradata7/cbam28.dbf12.1124.4111.824.1

This query will not return all tablespaces' datafiles, only those with at least a few KB of free space and ,thus, at least 1 row in DBA_FREE_SPACE.

Another query to sum this free space per tablespace:
  SELECT B.TABLESPACE_NAME,
         ROUND (SUM (B.BYTES) / 1024 / 1024 / 1024, 2) "SIZE GB",
         ROUND (SUM (B.MAXBYTES) / 1024 / 1024 / 1024, 2) "MAX SIZE GB",
         A.FREE_SPACE_GB "FREE SPACE GB",
         ROUND (
            (SUM (B.MAXBYTES - B.BYTES) / 1024 / 1024 / 1024) + A.FREE_SPACE_GB,
            2)
            "TOTAL FREE SPACE GB"
    FROM DBA_DATA_FILES B,
         (  SELECT TABLESPACE_NAME,
                   ROUND (SUM (BYTES) / 1024 / 1024 / 1024, 2) FREE_SPACE_GB
              FROM DBA_FREE_SPACE
             WHERE TABLESPACE_NAME IN ('CBAM')
          GROUP BY TABLESPACE_NAME) A
   WHERE     B.TABLESPACE_NAME IN ('CBAM')
         AND B.TABLESPACE_NAME = A.TABLESPACE_NAME
         AND B.MAXBYTES != 0
GROUP BY B.TABLESPACE_NAME, A.FREE_SPACE_GB
ORDER BY 1;

The output:
TABLESPACE_NAMESIZE GBMAX SIZE GBFREE SPACE GBTOTAL FREE SPACE GB
CBAM284.58309.3823.8648.66

Again, this query will not show you the tablespace's full size. In "SIZE GB" column are added the datafiles' sizes which have free space.