Riverbed Steelfusion – High Latency With IBM Storwize v7000 (BAW12)
As you may recall from previous blog posts, Riverbed Steelfusion is an interesting way to provide the performance of a local VMware environment to a remote office whilst also keeping the data replicated back to a central data centre (or even to a cloud environment).
This is done by presenting out storage from the data centre, via a Steelfusion Core (in the data centre), across your WAN connection to a Steelfusion Edge device. This storage is then presented as local storage to the virtual environment that is running on the Steelfusion Edge device. There are other types of configuration for the Steelfusion environment but that then doesn’t have as much appeal, in my eyes.
Anyway, recently we stumbled across a situation where we were seeing extremely large latency (especially with write latency) on the IBM Storwize v7000 storage which was being used for the presented storage in Steelfusion. When I’m saying extremely high latency, we’re talking about latency figures in the 1000ms range. This is obviously not good when any data that is modified at the remote office needs to write back to the data centre to be fully committed. With high latency like this, there is a strong possibility that a lot of this data will not be able to be committed and the amount of uncommitted data will increase.
After investigating this issue with Riverbed, we stumbled onto an issue that had been seen with an Equalogic SAN firmware where uncommitted data would increase significantly. After careful consideration of the potential impact of the following commands, we implemented them onto two presented LUNs. To our delight, this resulted in an immediate drop in the latency on those LUNs. After implementing the changes across all of the LUNs, we were happy to say that the latency returned to a normal level again.
The commands in question need to be run on the Steelfusion Core and need to be run for each presented LUN. Please note that these commands worked in our environment and are provided for information only. You should always raise a call with Riverbed or one of their partners to investigate this further before implementing.
To enable the fix (Disable ordered task tag):
storage lun modify serial <lun serial> serialize-write enable
or
storage lun modify alias <lun alias> serlialize-write enable
To revert the change:
no storage lun modify serial <lun serial> serialize-write enable
or
no storage lun modify alias <lun alias> serialize-write enable
The link to the Riverbed KB article is here (although you would need to log in to view this):
https://supportkb.riverbed.com/support/index?page=content&id=S23646
Blog-a-Week 2018 Posting (BAW):
In previous years, I have had periods throughout the year where I have been unable to post an entry to my blog. During 2018, my aim is to post a blog entry each week of the year. I’m hoping that the blog entries will all be useful bits of information or items that I’ve been working on recently but if the standard of the entries drops, I apologise now, as this could have been a busy week and I wasn’t able to put in as much time as I would have liked into the post