Oracle is moving away from ASMlib, and introducing ASM Filter Drivers as a replacement.
ASM Filter Drivers will handle consistent device naming and permissions, as well as filter out illegal IO to ASM devices to protect against rogue dd commands corrupting ASM disks.
Future plans include support for TRIM commands to enable thinly provisioned disks to reclaim deleted blocks without having to resort to the massively dangerous ASRU tool.
ASM Filter Drivers were introduced with Oracle 220.127.116.11, but the implementation is currently one massive kludge. By default on 18.104.22.168, OEL7 is not supported without a patch (patch 21053000). OEL6 UEK is also not supported without a patch (patch 18321597).
Note that the patches require OPatch 22.214.171.124, but Oracle Grid Infrastructrue 126.96.36.199 installs OPatch 188.8.131.52.3 so you have to patch the patcher (patch 6880880), so you can patch the Oracle software, to make Oracle ASM Filter Drivers work with Oracle’s own operating system kernel. Clear? Good!
You cannot install Filter Drivers by default. You have to migrate to them from UDEV or ASMlib.
Oracle 12.2 should hopefully fix this mess and make Filter Drivers actually usable, but in the meantime it might be fun to play with the new technology and see what it can do.
To make our lives simpler, and because I just don’t enjoy spending half my day downloading and installing patches, I am going to start from a supported standpoint.
Oracle OEL 6.4 Red Hat Kernel with Oracle 184.108.40.206.
I have four 400GB LUNs presented from an EMC Unity 600F all-flash array.
Because I cannot install ASM Filter Drivers by default, I have installed my set up my four disks with UDEV rules and created an ASM disk group on them called DATA.
I could have used ASMLib. But ASMLib is only installed by default with the UEK kernel, and that kernel is not supported for ASM Filter Drivers without some patch madness.
To enable ASM Filter Drivers, first we need to shut everything down. Log in as root:
[root@oel6solo ~]# $ORACLE_HOME/bin/crsctl stop has CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'oel6solo' CRS-2673: Attempting to stop 'ora.DATA.dg' on 'oel6solo' CRS-2673: Attempting to stop 'ora.LISTENER.lsnr' on 'oel6solo' CRS-2677: Stop of 'ora.DATA.dg' on 'oel6solo' succeeded CRS-2673: Attempting to stop 'ora.asm' on 'oel6solo' CRS-2677: Stop of 'ora.LISTENER.lsnr' on 'oel6solo' succeeded CRS-2677: Stop of 'ora.asm' on 'oel6solo' succeeded CRS-2673: Attempting to stop 'ora.evmd' on 'oel6solo' CRS-2677: Stop of 'ora.evmd' on 'oel6solo' succeeded CRS-2673: Attempting to stop 'ora.cssd' on 'oel6solo' CRS-2677: Stop of 'ora.cssd' on 'oel6solo' succeeded CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'oel6solo' has completed CRS-4133: Oracle High Availability Services has been stopped. [root@oel6solo ~]#
Make sure everything is down, and then you can configure ASM FD.
[root@oel6solo ~]# asmcmd afd_configure Connected to an idle instance. AFD-627: AFD distribution files found. AFD-636: Installing requested AFD software. AFD-637: Loading installed AFD drivers. AFD-9321: Creating udev for AFD. AFD-9323: Creating module dependencies - this may take some time. AFD-9154: Loading 'oracleafd.ko' driver. AFD-649: Verifying AFD devices. AFD-9156: Detecting control device '/dev/oracleafd/admin'. AFD-638: AFD installation correctness verified. Modifying resource dependencies - this may take some time. ASMCMD-9524: AFD configuration failed 'ERROR: OHASD start failed'
Great! An error. Luckily in this case you can ignore it.
[root@oel6solo ~]# $ORACLE_HOME/bin/crsctl start has CRS-4123: Oracle High Availability Services has been started.
Now log in as Oracle and source the GI environment.
Check that ASM Filter Drivers are running:
[oracle@oel6solo ~]$ asmcmd afd_state ASMCMD-9526: The AFD state is 'LOADED' and filtering is 'DEFAULT' on host 'oel6solo.comp-lab.sc'
We can now label our existing disks for ASM Filter Drivers.
[oracle@oel6solo ~]$ asmcmd afd_label DATA1 /dev/sdb --migrate [oracle@oel6solo ~]$ asmcmd afd_label DATA2 /dev/sdc --migrate [oracle@oel6solo ~]$ asmcmd afd_label DATA3 /dev/sdd --migrate [oracle@oel6solo ~]$ asmcmd afd_label DATA4 /dev/sde --migrate
Check that the disks appear as ASM filtered disks:
[oracle@oel6solo ~]$ asmcmd afd_lsdsk -------------------------------------------------------------------------------- Label Filtering Path ================================================================================ DATA2 ENABLED /dev/sdc DATA4 ENABLED /dev/sde DATA3 ENABLED /dev/sdd DATA1 ENABLED /dev/sdb
Check that the disks appear at /dev/oracleafd/disks
[oracle@oel6solo ~]$ ls -al /dev/oracleafd/disks total 16 drwxrwx---. 2 oracle asmadmin 120 Mar 3 13:52 . drwxrwx---. 3 oracle asmadmin 80 Mar 3 13:52 .. -rw-r--r--. 1 root root 9 Mar 3 13:52 DATA1 -rw-r--r--. 1 root root 9 Mar 3 13:52 DATA2 -rw-r--r--. 1 root root 9 Mar 3 13:52 DATA3 -rw-r--r--. 1 root root 9 Mar 3 13:52 DATA4
Since we started with UDEV rules, if we check our ASM configuration we will still see /dev/oracleasm as the ASM disk discovery string:
[oracle@oel6solo ~]$ srvctl config asm ASM home: Password file: +DATA/orapwasm ASM listener: LISTENER Spfile: +DATA/ASM/ASMPARAMETERFILE/registry.253.937659731 ASM diskgroup discovery string: /dev/oracleasm/disks
We need to switch ASM to use the ASM Filter Driver discovery string of AFD:*.
[oracle@oel6solo ~]$ asmcmd afd_scan [oracle@oel6solo ~]$ asmcmd dsset 'AFD:*' [oracle@oel6solo ~]$ $ORACLE_HOME/bin/asmcmd dsget parameter:AFD:* profile:AFD:*
At this point we can remove our UDEV rules and restart the UDEV service. Once that is done, the disks at /dev/oracleasm should disappear.
Finally enable ASM filtering on reboot:
[oracle@oel6solo ~]$ $ORACLE_HOME/bin/asmcmd afd_filter -e