Change Array to Group Scoped Targeting (HPE Nimble Storage)

The array OS 5.1.x and later supports iSCSI Group Scoped Target (GST) on iSCSI arrays (this is new), previous to this it was using VST Volume Scoped Targeting. GST reduces the number of individual host connections you need to configure and manage, which saves you time and also ensures you don’t hit limitations in the iSCSI implementation on the host side.

For example, with Volume Scoped Target (VST), if you need to connect four iSCSI volumes to the host, you would need to connect each target to the host individually. With GST, if you want to connect those same four volumes, you only need to connect to the one target.

On VMware for example, if you had a Nimble Storage array with 4 iSCSI IP addresses, and 10 datastores presented (and using VST) you’d see 4 targets for the array group, then 10 targets, one for each datastore. In this case they all appear with LUN ID 0 typically.

Now when you swap to GST you only see the 4 iSCSI IP addresses, and then each datastore is accessible via these group scoped targets instead.

This is a consequence of AS-117089, the Network Affinity Zones are no longer honoured, partly due to all connections now being meshed.

NOTE: When you perform a new installation of the array 5.1.x or later, your default target will use GST. If you upgrade to 5.1.x or later, your default target will continue to use VST. This is an important consideration, if you have an older array which was running a version pre-5.1.x then it would have come set with VST, if you’ve upgraded to 5.1.x or later you’ll still be using VST, unless of course you use the procedure below to swap it to GST.

https://infosight.hpe.com/InfoSight/media/cms/active/public/pubs_CLI_Administration_Guide_5_1_x.whz/index.html#pkd1536255247713.html

NOTE: For every volume you are changing you must offline its access, its recommended to totally drain the array of data (that’s how I did it), but if that is not possible, just ensure that there is no I/O on the volume you are changing or it will cause an outage.

Goal

The goal of these steps is to:

1. Swap Existing Datastores from FVST to GST – Swap all the existing volumes/datastores from FVST to GST, i.e. from Volume to Group Targetting. (if you are going to replace all the volumes/datastores you can skip this step).
2. Set Default Setting – Set the default setting to GST (Group Scope Targetting) for all new volumes/datastores that are created after that point.
3. Complete FVST to GST Swap – Complete a swap from FVST to GST to allow the array to be put back into service and storage presented once more.

Procedure

1. Disconnect all hosts (ESXi and Windows) from the volume(s) to be migrated.

2. Connect to the group CLI.

3. For each volume, verify the number of active connections is 0, with the following command:

vol --info [volumeName] | grep connections

4. Take each volume to be reconfigured offline with the following command (ensure its not got any IO!)

vol --offline [volumeName]

5. Remove the ACLs from the volume(s) For each volume, issue the command:

vol --info [volumeName] | egrep "Name|Initiator Group"

With this you see the list of Initiator Groups allowed access to the the volume, make note of these Initiator Groups listed, for later in the procedure.

For each Initiator group listed, issue the following command:

vol --removeacl [volumeName] --initiatorgrp [initiatorGroupName]

6. Once all ACLs have been removed from each volume, proceed to the next step.

7. Change the target scope: For each volume, issue:

vol --edit [volumeName] --iscsi_target_scope group

Verify the change:

vol --info [volumeName] | grep Scope

If it is correct you’ll see:

Nimble OS $ vol --info datastore001 | grep Scope
iSCSI Target Scope: iSCSI Group Target

Repeat until all your existing volumes are done.

IMPORTANT: It is highly recommended to ensure that each datastore uses a unique LUN ID, much like what you have on a fibre channel storage area network, ensuring that each datastore is presented with a unique LUN ID across your whole iSCSI (and Fibre Channel) estate ensures you don’t have a clash of LUN IDs, which can cause re-signaturing of VMFS volumes and outages (on VMWare) and other issues on NTFS etc.

8. Now change the default target scope, this ensures that all newly created and presented datastores are created with GST instead of VST.

group --edit --default_iscsi_target_scope group

Verify the change with:

Nimble OS $ group --info | grep -i scope
Default iSCSI Target Scope: iSCSI Volume Target
Encryption default settings scope: volume

< Post Change >

Nimble OS $ group --info | grep -i scope
Default iSCSI Target Scope: iSCSI Group Target
Encryption default settings scope: volume

NOTE: that the “Encryption default settings scope” being set to “volume” is as expected, this value does not need to be changed, it defines the scope of where the encryption of a volume is, i.e. scoped to the volume itself.

Change iSCSI Default Target Scope
https://infosight.hpe.com/InfoSight/media/cms/active/public/pubs_CLI_Administration_Guide_5_1_x.whz/index.html#lgs1579039268104.html

Change iSCSI Volume Scope from VST to GST
https://infosight.hpe.com/InfoSight/media/cms/active/public/pubs_CLI_Administration_Guide_5_1_x.whz/index.html#hme1536256896246.html

Change iSCSI Volume Scope from GST to VST
https://infosight.hpe.com/InfoSight/media/cms/active/public/pubs_CLI_Administration_Guide_5_1_x.whz/index.html#zow1536259763236.html

9. Re-add the ACL(s) to the volume(s), refer to the previously-noted list of each volume’s Initiator Group ACLs you got earlier in step 5, remember to set the unique LUN ID!

For each Initiator Group listed, on each volume, issue the command:

vol --addacl [volumeName] --initiatorgrp [initiatorGroupName]

NOTE: Ensure you add the VLAN ID argument to the end to ensure you are presenting the volume with the correct ID.

10. Once all ACLs have been re-added to each volume, you can then bring them all back on line with:

vol --online [volumeName]

11. Re-establish connections to the volumes by performing a rescan from the host(s), in the case of VMware you’ll no longer see a target for each volume (plus the 4 for each iSCSI IP), you’ll now see only 4 targets, because they are now working behind this “group scope”.