Fishworks team member Eric Schrock defines shadow migration - one of the major new features in the Sun Storage 7000 2009.Q3 software release. Schrock explains this project's conception, development, and birth in a blog entry that outlines the feature's choice of migrating via interposition versus synchronization, and provides an overview of how shadow migration works.

The basic design requirements he and fellow Fishworks engineer Adam Leventhal had were:

• Migration over standard data protocols (NFS) from arbitrary data sources without the need to have special software running on the source system.
• Migrated data must be available before the entire migration is complete, and must be accessible with native performance.
• All the data to migrate the filesystem must be stored within the filesystem itself, and must not rely on an external database to ensure consistency.

What the Fishworks engineers figured out was that they could create a 'shadow' filesystem that could pull data from the original source if necessary, but then fall through to the native filesystem for reads and writes once the file had been migrated. Another interesting tidbit was that they could leverage the NFS client on Solaris and do this entirely at the VFS (virtual filesystem) layer, allowing for the migration of data between shares locally or possibly even over other protocols without changing the interpositioning layer.

"The other nice thing about this architecture is that the kernel remains ignorant of the larger migration process," Schrock writes. "Both synchronous requests (from clients) and background requests (from the management daemon) appear the same. This allows us to control policy within the userland software stack, without pushing that complexity into the kernel. It also allows us to write a very comprehensive automated test suite that runs entirely on local filesystems without need a complex multi-system environment."

Schrock follows this with a quick comparative between migrating via synchronization and external interposition, with the final decision for shadow migration to use interposition. However, there is a difference. Shadow migration is integrated into the appliance and doesn't require a separate physical machine, unlike external interposition.

He explains, "When shares are created, they can optionally 'shadow' an existing directory, either locally ... or over NFS. ... Shadow migration (is) implemented using on-disk data within the filesystem, so there is no external database and no data stored locally outside the storage pool. If a pool is failed over in a cluster, or both system disks fail and a new head node is required, all data necessary to continue shadow migration without interruption will be kept with the storage pool."

Drawbacks of shadow migration: it requires a commitment before the data has finished migrating and portions of the data exists in two locations during migration, which means that backups are more complicated and snapshots may be incomplete and/or exist only on one host.

More Information

What is Shadow Migration? - Schrock's blog entry

Sun Enhances Its Unified Storage Systems

Sun Storage 7000 Unified Storage Systems

Sun Executives Discuss Enhancements to the Sun Unified Storage Systems - video

Sun Storage Unified Storage Family Data Sheet

Press Release: "Sun Microsystems Boosts Performance Across Industry-Leading Unified Storage Portfolio and Trumps Industry with New Standards for Data Protection" [...read more...]

		fullsource
Print Email Current Profile Archive Blog Generate newsletter