basic ability to research OUIs based on CSV input of WWNs such as a Nickname file or a Missing-Nickname output.
This script basically uses “The DNS Method” on a list of input WWNs to determine a list of unique OUIs for reference, looks each up, and adds that information to the output stream.
When some customers look at the output of our Hardware Probes such as the 8g FC8, and they’re confirming that all their Oracle transactions are meeting the SLA of 8ms, the LUNs they see are a bit unusual. They’re used to seeing LUNs such as LUN32, LUN33, but VirtualWisdom shows them LUNs like 17652, and they just don’t match up. This hinders the utility of the data, and may reduce the confidence they have in the data itself. When data drives decisions, we need accuracy and we need to be as easy to use as we can based on what we have available on the FC link.
The truth of the mismatch is that the devices involved actually convert the LUN numbers: where Virtual Wisdom shows you “17652″, that’s actually the LUN “on-the-wire”. The actual LUN in the SCSI frame is that large number, but vendor-specific tools convert it to manageable, familiar numbers — which can make the actual LUN appear “wrong”. Although it’s very simple to say “VirtualWisdom isn’t making the same conversion as your vendor-specific tools”, we’d rather be more helpful. When you have a problem on your SAN, “VirtualWisdom doesn’t support…” offers no help towards fixing the problem. Rather than “17652″, we’d rather tell you “Symm Device 1E2B”, which — for a very important customer of ours — is a comfortable middle-ground in identifiers and terms for the LUNs.
So how can we get a usable label on those LUNs? How do we do the lookups for you so that in an emergency, you have the details you need, already dereferenced?
As you’ve seen in the articles I’ve posted, I’m a Field Application Engineer for Virtual Instruments, and making this conversion — plus automating it — are the challenges I enjoy working on and sharing. This how-to article shares how to get SymmDevice aliases mapped onto LUNs as nicknames in a scriptable method. Let’s dive in:
Process Overview
For this process, our flow will look like the following:
In this diagram, the “sysinq” command is used to generate the “sysinq.txt” file; that part of the process is quite dependent on the tools available on your servers, hence shown dotted.
Overview
We at Virtual Instruments have witnessed this odd mismatch in LUNs for some time, and initially we just said “Subtract 0×4000″, but that didn’t work precisely. In another instance, we found two servers with a very similar number, so the informal rule became “subtract 17506″, but we later found that this rule was unusable outside that customer. Recently, in very detailed discussions, a customer found the logic that gets them to their LUN Nicknames, as follows:
The “sysinfo” file looks like this: (mocked-up example from testcases)
disk 1071 8/0/12/1/0.140.0.54.6.1.4 sdisk CLAIMED DEVICE EMC SYMMETRIX
lunpath 1071 8/0/12/1/0.0x50060482d5123456.0x4392000000000000 eslpt CLAIMED LUN_PATH LUN path for disk6219
disk 1071 8/0/12/1/0.140.0.54.6.1.4 sdisk CLAIMED DEVICE EMC SYMMETRIX
disk 1071 8/0/12/1/0.140.0.54.6.1.4 sdisk CLAIMED DEVICE EMC SYMMETRIX
lunpath 1071 8/0/12/1/0.0x50060482d5123456.0x4392000000000000 eslpt CLAIMED LUN_PATH LUN path for disk6219
lunpath 1071 8/0/12/1/0.0x50060482d5123456.0x4392000000000000 online
The key part of this file is the line that says:
lunpath ...0x50060482d5123456.0x4392... disk6219
In this case, “50060482d5123456″ is the storage device’s WWN, and 0x4392 is the LUN (17298 in decimal) we detect as the actual LUN used in the SCSI exchange for disk6219.
The “syminq” file looks for a certain server like this: (again, mocked-up example from testcases)
/dev/rdisk/disk6219 R1 000190102037 5773 3701E8B000 2096640
/dev/rdsk/c23t1d2 R1 000190102037 5773 3701E8B000 2096640
/dev/rdsk/c43t1d2 R1 000190102037 5773 3701E8B000 2096640
/dev/rdsk/c133t14d5 R1 000190102037 5773 3701E8B000 2096640
The key part of this file is the line that says:
...disk6219 ...3701E8B...
That rdisk entry is as follows: (keep in mind, the content is replaced by bogus values: some common values may not align anymore)
I don’t have official information, but it’s possible that 01E8B is actually “Symmetrix device 01E”, and “Director #8B”. 0x8b larger than 16 used to mean Processor B, but that seems a bit outdated now. The tools we use could easily give out 5-digit SymmDevice-XXXX values if we could confirm this numbering breakdown.
Essentially, the SysInfo2LUNNickname.awk script does the following:
syminq file to provide “better” nicknames where possiblesysinfo to realize nicknames, substituting syminq results where possibleThe resulting nicknames.csv has a series of lines such as:
"LUN","50060482d5123456","17296","disk6217"
"LUN","50060482d5123456","17297","disk6218"
"LUN","50060482d5123456","17298","SymmDevice-1E8B"
"LUN","50060482d5123456","17299","disk6220"
As you can see, where there is a “better” match, “SymmDevice-1E8B” is used; otherwise, “diskXXXX” is still there. At this particular customer, “1E8B” is an example of a common name for their LUNs rather than the 17298 reported by VirtualInstruments or the 0×4392 reported in hex.
For exceptionally large syminq files, because gawk.exe is loading up a large associative array, memory usage may temporarily peak; this has been tested on 500k – 1MB text files without visible adverse effects, but hidden memory usage such as interpreters’ associative arrays and implicit memory-management is something to be aware of.
Running the Script
gawk.exe -v SYMINQ=syminq.txt -f SysInfo2LUNNickname.awk sysinfo.txt > nicknames.csv
It’s that simple. In order to improve re-use, I collected this logic as the AWK script because I’ve never had portability problems with AWK except for the UNIX/Windows CR/CRLF/LF text line-endings debate.
The script is non-interactive, but prints the results, so the script is typically run with the output redirected to a file. Running it is very quiet, as you’d expect:
Complete Example
A complete example is difficult for this how-to because the files used are scattered on each server; collecting the sysinfo, and the syminq output from each server may be the more difficult part. My test example looks like the following (I always recommend using full pathnames for tools):
@echo off
cd \VirtualWisdomData\
REM following is all on one line but split here for easier reading
C:\UnxUtils\usr\local\wbin\gawk.exe
-v SYMINQ=\sandata\syminq01.txt
-f \sandata\SysInfo2LUNicknames.awk
\sandata\sysinfo > \VirtualWisdomData\DeviceNickname\nicknames.csv
Similar to previous Nickname-generation/derivation tutorials, this process generates a single nicknames.csv file. You could append this result to any other generated file for which you already have an import schedule, or create a new one. In order to be as complete as possible, I’ve included an import schedule example that should be surprisingly similar to the others (and similarly brief: the User Guide has more detail regarding Schedules)
\DeviceNickname directory of your VirtualWisdomData folder:
Virtually all SAN devices that are zoned for traffic have names (in fact, if you have nicknames/aliases in your zone files, then you can directly convert zone info to nicknames). VirtualWisdom’s filtering capabilities allow you to restrict a Dashboard, Report, or Alarm Policy Ruleset to a specific datacenter or business unit, but often creating those UDCs can be cumbersome.
A recent customer created UDCs with 192 values across three metric sets, allowing him to group data by specific servers, storage, and virtualizers automatically; this “how-to” is intended to show how you can do the same.
Process Overview
For this process, we need only a set of nicknames; the simpler old-format nickname file looks like:
500604825D2E2144,"DMX1911_FA3AA"
2100001B329FE31D,"Billing44_HBA0"
10000000C741ABCD,"4241_7b1"
(Notice: WWN first, no spaces, optional quotes for safety)
Generate UDC Values from Nickname Pattern
Our flow for this process or pipeline looks like the following diagram:
The tool we use here is “awk”, or “awk.exe”, or “gawk.exe”; in Solaris, look for “nawk”. It’s on virtually every non-Windows system, Microsoft has a version in its tools for UNIX, or Google may help you find a copy. As well, UnxUtils has a version.
Awk is an interpreter, so needs a script or program, and for that, we use TransformUDC.awk which takes the following parameters:
| parameter | Meaning |
|---|---|
| COL | What column in the CSV input is the Nickname? (default: 1) |
| NAMEFCX_LINK | Name of the ProbeFCX::Link UDC |
| NAMEFCX_SCSI | Name of the ProbeFCX::SCSI UDC |
| NAMEFCX_SCSIINIT | Name of the ProbeFCX::SCSI UDC, matching Initiators only |
| NAMEFCX_SCSITARG | Name of the ProbeFCX::SCSI UDC, matching Targets only |
| NAMESW | Name of the ProbeSW::Link UDC (default: Transformed_UDC) |
| TRANSFORM | Transform (basically the ‘s/x/y/g’ in a “sed -e ‘s/x/y/g’” command) (default: remove last two _sect_sect: DC_Serv1_fcs0_SW12P121 –> DC_Serv1) |
| UDCDEFAULT | Default value for UDCs (default: Unknown) |
A simple command such as the following will generate our results:
gawk.exe -f TransformUDC.awk Nicknames.csv
In our nickname file, the older format (first and second variant up to VW-3.1) gave us the nickname as the second parameter, so let’s tell the script that the nickname is in column #2:
gawk.exe -v COL=2 -f TransformUDC.awk Nicknames.csv
The problem is: how do we want the UDC values defined? If you’ve used “sed” or “awk” before, there’s a basic replacement term that looks like s/dog/cat/g or gsub("dog","cat",$0) … this part really depends on your nickname format, but looking above, we have nicknames that look like:
500604825D2E2144,"DMX1911_FA3AA"
500604825D2E2145,"DMX1911_FA4AA"
500604825D2E7744,"DMX1927_FA3AA"
500604825D2E7745,"DMX1927_FA4AA"
500604825D2E7747,"DMX1927_FA6AA"
500604825D2E7748,"DMX1927_FA7AA"
500604825D2E774C,"DMX1927_FA13AA"
500604825D2E774D,"DMX1927_FA14AA"
2100001B329FE35D,"Billing43_HBA0"
2100001B329FE35E,"Billing43_HBA1"
2100001B329FE31D,"Billing44_HBA0"
2100001B329FE31E,"Billing44_HBA1"
10000000C741ABCD,"4241_7b1"
We see how, in this example, chopping off everything after the “_” gives names such as “Billing43″ and “DMX1927″. In see and awk, we would write: s/_.*$//g so we’ll use that as our transform. How can we test this?
Trim Quotation Marks
We could trim off the quotation marks around the second field using this: (“,” as field-separator, convert (“) to (), an empty replacement)
gawk -F, '{gsub("\"","",$2); print; }' Nicknames.csv
… unfortunately, we need to use quotation marks for the script, and then we need a bunch of “\” escape sequences, so it looks much more complex running it:
gawk.exe -F, "{gsub(\"\\\"\",\"\",$2); print; }" Nicknames.csv
… which looks like:
Truncate All After “_”
Based on the example above, we can now test whether our transform (“s/._*$//g”, or gsub(“._*$”,””,…) ) gives us the results we want, such as (notice: “print $2″, so we’ll only see the second field):
gawk.exe -F, "{gsub(\"\\\"\",\"\",$2); gsub(\"_.*$\",\"\",$2); print $2; }" Nicknames.csv
… which looks like:
This means our transform works, so let’s use it in the script:
gawk.exe -v TRANSFORM="s/_.*$//g" -v COL=2 -f TransformUDC.awk Nicknames.csv
Unfortunately, “4241″ is not worthwhile to us because it’s only one matching name, so let’s trim that one off by saying “minimum of 2 matching names per UDC value”:
gawk.exe -v MIN=2 -v TRANSFORM="s/_.*$//g" -v COL=2 -f TransformUDC.awk Nicknames.csv
Finally, What do we want to call the UDC? The tool always generates a ProbeSW::Link UDC, and if unnamed, defaults to “Transformed_UDC”. The Name of the UDC is limited to 32 characters, and values themselves to 24 characters; the name of the UDC becomes the name of the “metric” or context that we are generating. Suppose while working at XYZ Cheese and Dairy Distributors, we want a UDC called xyz-SW-BizUnit (we need to use “_” rather than “-”):
gawk.exe -v NAMESW="xyz_SW_BizUnit" -v MIN=2 -v TRANSFORM="s/_.*$//g" -v COL=2 -f TransformUDC.awk Nicknames.csv
Let’s run this with a redirection (“>”) to store the results to a file:
gawk.exe -v NAMESW="xyz_SW_BizUnit" -v MIN=2 -v TRANSFORM="s/_.*$//g" -v COL=2 -f TransformUDC.awk Nicknames.csv > \VirtualWisdomData\UDCImport\xyz-UDCs.udc
Running this as a command in cmd.exe is relatively quiet because this is a non-interactive command. It tends to look like the following:
Importing this example, we see the following (you’ll note: the default has also been set using “-v UDCDEFAULT=Other”) :
Schedule UDC Import
Creating the schedule is relatively straight-forward: although there is some strong guidance in the VirtualWisdom User Guide, a complete example would like like the following:
\UDCImport\ directory of your VirtualWisdomData folder:
The benefit here is that the UDC always replaces existing values without prompting. As well, after import, a UDC re-calculates values for both past summaries and new summaries. This allows you to “fix history” if your UDC is not quite correct the first time.