There have been a few issues with ASM during this exercise of adding nodes; ASM Instance Number incorrect, and Disk partitions not shared across all nodes -- actually has happened twice but for 2 different reasons.
Issue 1: ASM Instance Number Incorrect. The ASM instances are using an spfile
The x is replaced with the number of the ASM instance.
Once the configuration is set in DBCA, the utility asks this is an ASM database would you like to extend the ASM instance? Of course, the answer is yes or there will be problems when the database instance is extended -- Disks will not be available.
However, this bombs with the Instance Number is incorrect. It seems that the spfile that DBCA decides to use is a generic version
That's an spfile without the instance number attached. Since you can't edit the spfile directly, back on the original install host create a pfile from the spfile while logged into the ASM instance
create pfile from spfile;
I'm good with the location, but if you want to specify the directory and filename that can be done with
create pfile='/oracle/products/dbs/init+ASM1.ora' from spfile;
Now edit the pfile, I add the new instance number as well as any instance numbers that maybe missing. Then recreate the spfile file.
create spfile from pfile;
Then I copy this file over to the new nodes spfile+ASM.ora generic file.
Rerun DBCA and the ASM instance is extended, as does the database.
Issue 2a: Disk Partitions not shared across all nodes
The first time the message was received only one of the partitions wasn't being seen. The fix:
1. Ensuring that Oracle was in the Disk group (sets the permissions to access the disks)
2. Check the raw devices in the /raw/dev file to ensure they are correctly specified and not duplicated. This
will be on the new node.
And in case anyone is wondering how I did 1 and 2, I'll have to defer to my boss as he's actually the one that handles the raw devices and ensuring the powerpath ids all match up.
Issue 2b: Disk Partitions not shared across all nodes and all of the diskgroups are listed
Since this is my fourth node and this is the only one I have had this issue with I'm inclined to believe it has something to do with how I created the spfile and the pfile that points to the spfile. So tomorrow I get to experiment on how to correct this situation. My goal is to correct it, without having to bounce any of the ASM instances.
So tomorrow should be an interesting day.
I still don't have a solution to this problem. I did open a SR with Oracle which they haven't provided any useful information to try only asked for information that was already provided and files that were already uploaded. I often wonder if the support analyst read the actual information or just make assumptions so that they can push the issue back to the customer and get the clock off of them.