Do Not Mix Oracle ASM Disks with LVM

Tech note (FAQ)

Question

This tech note discusses Oracle ASM and LVM disks

Answer

In Oracle 10g Oracle released a disk management layer named Automatic Storage Management (ASM). This is typically deployed with Oracle RAC in order to manage raw disks used by the Oracle database in a RAC cluster.

Disks being used by Oracle ASM cannot be also used with AIX LVM. The main reason for this is because the Oracle ASM software puts information on the raw disk to identify it as an Oracle disk, which wipes out any existing PVID or VGDA information.

If an attempt is made to use LVM on a disk already in an Oracle ASM disk group the following errors will be seen:

root> extendvg -f apps hdiskpower7
0516-1339 extendvg: Physical volume contains some 3rd party volume group.
0516-1397 extendvg: The physical volume hdiskpower7, will not be added to the volume group.
0516-792 extendvg: Unable to extend volume group.


root> mkvg -y testvg hdiskpower7
0516-1339 mkvg: Physical volume contains some 3rd party volume group.
0516-1397 mkvg: The physical volume hdiskpower7, will not be added to the volume group.
0516-862 mkvg: Unable to create volume group.


Here is the debug output from the alog LVMT debug:

[S 1507490 1531914 07/10/09-07:48:58:138 mkvg.c 412] extendvg apps hdiskpower7
[2 1507490 0:024 mkvg.c 1662] validate_pvs: Found an Oracle disk(hdiskpower7)
[2 1507490 0:024 mkvg.c 1668] validate_pvs: Disk(hdiskpower7) is in use by Oracle or f flag not used [1 1507490 0:024 mkvg.c 1870] validate_pvs: FAIL: Invalid PVs! num_invalid_pvs rc=1
[1 1507490 0:024 mkvg.c 686] main: FAIL: validate_pvs failed
[1 1507490 0:024 mkvg.c 167] cleanup_exit: FAIL: pv_failures=1
[E 1507490 0:292 mkvg.c 268] extendvg: exited with rc=1

LVM knows to search for the Oracle identifier on these disks, and will fail the operation. You can manually check for the identifier by running the lquerypv command.

# lquerypv -h /dev/hdiskpower7
00000000 00820101 00000000 80000000 F7BE76D7 |..............v.|
00000010 00000000 00000000 00000000 00000000 |................|
00000020 4F52434C 4449534B 00000000 00000000 |ORCLDISK........|
00000030 00000000 00000000 00000000 00000000 |................|
00000040 0A100000 00000103 44415441 5F303030 |........DATA_000|
00000050 30000000 00000000 00000000 00000000 |0...............|
00000060 00000000 00000000 44415441 00000000 |........DATA....|
00000070 00000000 00000000 00000000 00000000 |................|
00000080 00000000 00000000 44415441 5F303030 |........DATA_000|
00000090 30000000 00000000 00000000 00000000 |0...............|
000000A0 00000000 00000000 00000000 00000000 |................|
000000B0 00000000 00000000 00000000 00000000 |................|
000000C0 00000000 00000000 01F64610 3B306C00 |..........F.;0l.|
000000D0 01F64610 3B9C7C00 02001000 00100000 |..F.;.|.........|
000000E0 0001BC80 0000C800 00000002 00000001 |................|
000000F0 00000002 00000002 00000000 00000000 |................|

There are 3 data points to note in the lquerypv output:

1. The presence of the Oracle ASM identifier "ORCLDISK".

2. The lack of a PVID on location 80.

3. The lack of the AIX disk identifier "C9C2D4C1" in location 00.