How To Setup/Configure 11.2.0.X DBFS Filesystem On RAC Configurations CRS Managed (Non-Exadata) For Golden gate

How To Setup/Configure 11.2.0.X DBFS Filesystem On RAC Configurations CRS Managed (Non-Exadata) For Golden gate

This document provides an example about how to setup, configure and implement a CRS(RAC-cluster) managed DBFS filesystem on Non-Exadata configurations For Golden Gate

Oracle Database File System (DBFS) creates a standard file system interface on top of files and directories that are stored in database tables. Oracle DBFS is similar to the Network File System (NFS) protocol in that it provides a shared network file system that looks like a local file system.

Steps to implement a DBFS filesystem on a RAC/Cluster configuration (CRS managed) For Golden Gate:

Note: The following implementation was performed on a 2 node RAC cluster Oracle Server 11.2 release.

1.        First of all, fuse fuse-libs kernel-devel needs to be installed & configured on both  the nodes:

With root user.

# yum install fuse fuse-libs kernel-devel.

2.       Add the user_allow_other option in the /etc/fuse.conf file on both  the nodes:

With root user.

#echo user_allow_other > /etc/fuse.conf

#chmod 644 /etc/fuse.conf

#cat /etc/fuse.conf

user_allow_other

#ls -l /etc/fuse.conf

-rw-r--r-- 1 root root 17 Aug 12 21:10 /etc/fuse.conf

3.       Create the DBFS mount point on both nodes:

              # mkdir -p /oracle/dbfs_direct

              #chown oracle:dba /oracle/dbfs_direct

              #ls -ld /oracle/dbfs_direct

              drwxr-xr-x 2 oracle dba 4096 Aug 17 21:13 /oracle/dbfs_direct

              #su - oracle

              $ cd /oracle/dbfs_direct

              $ pwd

               /oracle/dbfs_direct

4.       Create the DBFS database user and the DBFS database repository from any node as follows:

Sqlplus / as sysdba

SQL> create bigfile tablespace dbfs_ts

          datafile '+DATA' size 1024M

          autoextend on

          next 100M

          maxsize 3G

          NOLOGGING

          EXTENT MANAGEMENT LOCAL

          AUTOALLOCATE

          SEGMENT SPACE MANAGEMENT AUTO;

Tablespace created.

SQL>

SQL> create user dbfs_user identified by dbfs_user

default tablespace dbfs_ts

quota unlimited on dbfs_ts; 2 3

User created.

SQL>

SQL> grant create session, create table, create view, create procedure,

dbfs_role to dbfs_user;

Grant succeeded.

SQL>

5.       Please execute bellows  from any one Node .

A.      Then execute the following commands through the database instance (e.g. GGTEST1) as the dbfs_user database user (from Node #1 only):

$ sqlplus dbfs_user/dbfs_user

SQL> show user

USER is "DBFS_USER"

SQL>

SQL> start dbfs_create_filesystem dbfs_ts FS1

Above command should be complete with No error.

B.      Set the NOCACHE storage option to the LOB segment used by DBFS, connect to the database instance (e.g. GGTEST1) “/as sysdba” (from Node #1 only) and execute the following commands as follows:

$ sqlplus "/as sysdba"

SQL> ALTER TABLE DBFS_USER.T_FS1 MODIFY LOB (FILEDATA) (NOCACHE LOGGING);

Table altered.

SQL> SELECT owner

, table_name

, segment_name

, logging

, cache

FROM dba_lobs

WHERE tablespace_name='DBFS_TS';

OWNER            TABLE_NAME          SEGMENT_NAME      LOGGING             CACHE

------------------------------ ------------------------------ ------------------------------ ------- ----------

DBFS_USER          T_FS1                   LOB_SFS$_FST_1    YES                       NO

6.       Then configure fuse library links on Both the nodes (connected as root OS user):

# echo "/usr/local/lib" >> /etc/ld.so.conf.d/usr_local_lib.conf

# cd /usr/local/lib

#ln -s $ORACLE_HOME/lib/libclntsh.so.11.1

#ln -s $ORACLE_HOME/lib/libnnz11.so

# locate libfuse.so

/lib/libfuse.so.2

/lib/libfuse.so.2.7.4

/lib64/libfuse.so.2

/lib64/libfuse.so.2.7.4

/usr/local/lib/libfuse.so

/usr/local/lib/libfuse.so.2

#ln -s /lib64/libfuse.so.2 libfuse.so

# ldconfig

# ln -s $ORACLE_HOME/bin/dbfs_client   /sbin/mount.dbfs

7.       As a following step, please download the latest version of the “mount-dbfs.sh” script from the following document:

 Configuring DBFS on Oracle Database Machine [ID 1591515.1]

File Name:- mount-dbfs-20160215.rar

Contains: - mount-dbfs.sh and mount-dbfs.conf

8.       Then edit the mount-dbfs.conf script and update the next parameters with the correct/respective values and place it to /etc/oracle/mount-dbfs.conf location on both nodes.

            DBNAME= GGTEST

            MOUNT_POINT= /apps/oracle/dbfs_direct

            DBFS_USER= dbfs_user

            ORACLE_HOME= /apps/oracle/product/11.2.0.4/dbhome_1

            DBFS_PASSWD= dbfs_user

# ls -lrth /etc/oracle/mount-dbfs.conf

Permission and ownership

-rw-r--r-- 1 root oinstall 3.7K Aug 23 13:37 /etc/oracle/mount-dbfs.conf

9.       Place the mount-dbfs.sh script at the “<Grid Infrastructure _HOME>/crs/script/” directory (on both the nodes):

#cd /apps/infra/product/11.2.0.4/dbhome_1/crs/script/

Permission and ownership on both nodes

#ls -lrth mount-dbfs.sh

-rwxr-xr-x 1 grid oinstall 13K Aug 31 15:30 mount-dbfs.sh

chown grid:oinstall /apps/infra/product/11.2.0.4/dbhome_1/crs/script/mount-dbfs.sh 

chmod +x /bin/fusermount 

Change dos to Linux on both node

#dos2unix mount-dbfs.sh

10.   Create a script to add dbfs mount resource .

cat add-dbfs-resource.sh

##### start script add-dbfs-resource.sh

#!/bin/bash

ACTION_SCRIPT=/apps/infra/product/11.2.0.4/dbhome_1/crs/script/mount-dbfs.sh 

RESNAME=dbfs_mount

DBNAME=GGTEST

DBNAMEL=`echo $DBNAME | tr A-Z a-z`

ORACLE_HOME=/apps/infra/product/11.2.0.4/dbhome_1

PATH=$ORACLE_HOME/bin:$PATH

export PATH ORACLE_HOME

crsctl add resource $RESNAME \

-type local_resource \

-attr "ACTION_SCRIPT=$ACTION_SCRIPT, \

CHECK_INTERVAL=30,RESTART_ATTEMPTS=10, \

START_DEPENDENCIES='hard(ora.$DBNAMEL.db)pullup(ora.$DBNAMEL.db)',\

STOP_DEPENDENCIES='hard(ora.$DBNAMEL.db)',\

SCRIPT_TIMEOUT=300"

##### end script add-dbfs-resource.sh

11.   Run script with oracle user.

$./add-dbfs-resource.sh

12.    Start dbfs resource from grid user it will start dbfs mount point on both node.

$crsctl start resource dbfs_mount

            CRS-2672: Attempting to start 'dbfs_mount' on ora-rac-105-lq

            CRS-2672: Attempting to start 'dbfs_mount' on ora-rac-106-lq

            CRS-2676: Start of 'dbfs_mount' on ora-rac-105-lq succeeded

CRS-2676: Start of 'dbfs_mount' on ora-rac-106-lq succeeded

Node1#

[oracle@ora-rac-105-lq script]$ df -h|grep /apps/oracle/dbfs_direct

dbfs-dbfs_user@:/     3.0G  120K  3.0G   1% /apps/oracle/dbfs_direct

[oracle@ora-rac-105-lq script]$

Node2#

[oracle@ora-rac-106-lq ~]$ df -h|grep /apps/oracle/dbfs_direct

dbfs-dbfs_user@:/     3.0G  120K  3.0G   1% /apps/oracle/dbfs_direct

[oracle@ora-rac-106-lq ~]$

Now Go and create files in this mount point from nodes.

[oracle@ora-rac-105-lq script]$ cd /apps/oracle/dbfs_direct/FS1/

[oracle@ora-rac-105-lq FS1]$ touch node1.txt

[oracle@ora-rac-106-lq ~]$ cd /apps/oracle/dbfs_direct/FS1/

 [oracle@ora-rac-106-lq FS1]$ touch node2.txt

[oracle@ora-rac-106-lq FS1]$ ls -lrth

total 0

-rw-r--r-- 1 oracle oinstall 0 Sep 14 11:54 node1.txt

-rw-r--r-- 1 oracle oinstall 0 Sep 14 11:54 node2.txt