A Previous Post quotes Robin Harris of ZDnet (Storage Mojo).
Edit (04-jun-97): George Santayanda on his storage sanity blog writes on the Flashdance. Cites 80%/pa reduction in flash prices, break-even wit HDD on 2010/11. He's been in storage for years and is a senior manager. And doesn't take things too seriously.
And he points to a Powerpoint by Jim Gray on the Flash/Solid State disk. Worth the read.
Storage Price TrendsThe Yr/Yr ratios are used for forward projections.
The 'Est Flash' column uses the current 'best price' (MSY) for flash memory and a Yr/Yr ratio of 3.25.
Flash memory is now very much cheaper than RAM - forward projections not done.
|Year||RAM $/Gb||Flash $/Gb||Est Flash $/Gb||Disk$/Gb||DVD $/Gb||Max flash||Max Disk|
Depending on the"Year-on-Year" ratio you choose for the reduction in $/Gb of Flash memory, and if you think both flash and disk drives will continue their plunge down the price curve, solid state memory (flash) may be the cheapest form of storage in under 5 years.
New Storage Organisation
Backups and ArchivesWith the price of large, commodity disk drives driving down near DVD's, and probably overtaking, within 5 years - and that's ignoring the cost of optical drives and the problems of loading the data you want. Why would you not want to store backups and archives on disk?
For safe backups, the disks cannot be in the same machine, nor actually spinning. If the disk is in a USB enclosure, this meaning being able to spin it down by command.
Small businesses can effect a safe, effective off-site backup/archive solution by pairing with a friend and using 'rysnc' or similar over the Internet (we all have DSL now, don't we?) to a NAS appliance. The NAS does need to store the data encrypted - which could be done at source by 'rsync' (yet another option) or created by using an encrypted file system and rsync'ing the raw (encrypted) file. Best solution would be to have the disks spin-down when not being accessed.
This technique scales up to medium and large businesses, but probably using dedicated file-servers.
And the same technique - treat disks like removable tapes - applies. If drives are normally kept powered down, the normal issues of wearing out just won't arise. Rusting out might prove to be an issue - and little effects like thermal shock if powered up when very cold.
Speed, Space and Transfer RateRobin Harris of ZDnet etc writes on Storage. He's flagged that as commodity disks become larger, a few effects arise:
- single-parity RAID is no longer viable, especially as disk-drive failure is correlated with age. Older drives fail more often. The problem is the time to copy/recreate the data - MTTR, and the chance of another failure in that window whilst unprotected.
- Sudden drive failure is the normal mode. Electronics or power supply dies - game over.
- Common faults in a single batch/model are likely to cause drives to fail early and together, and
- RAID performance is severely impacted (halved) when rebuilding a failed drive.
This single figure of merit is half the equation: The other side is "time to copy".
That scales with transfer time, size, on-board cache and sustained read/write speeds.
What the World Needs Now - a new filesystem or storage applianceJust as disk drives are bashing up against some fundamental limits - bigger is only bigger, not faster - Flash memory is driving down in price - into the same region as disks. [And nobody knows when the limits of the magnetic recording technology will be reached - just like the 'heat death' of Moore's Law for CPU speed in early 2003.]
Flash suffers from some strong limitations:
- Not that fast - in terms of transfer rate
- Asymmetric read and write speeds (5-10:1)
- bits wear out. Not indefinite life.
- potentially affected by radiation (including cosmic rays)
"Plan 9", the next operating system invented by the group who designed Unix (and hence Linux), approached just this problem. Files were stored on dedicated "File Servers" - not that remarkable.
Their implementation used 2 levels of cache in front of the ultimate storage (magneto-optical disk). The two levels of cache were memory and disk.
The same approach can be used today to integrate flash memory into appliances - or filesystems:
- large RAM for high-performance read caching.
- large, parallel flash memories for read buffering and write caching
- ultimate storage on disk, and
- archives/snapshots to off-line disk drives.
The Flash memory has to be treated as a set of parallel drives - and probably with parity drive(s) as well.
This arrangement addresses the write performance issues, leverages the
faster read speed (when pushing cache to disk) and mitigates the effect
of chip failure, bits wearing out and random 'bit-flips' not detected
and corrected internally.
They only deep question is: What sort of flash drives to use?
- compact flash are IDE (ATA) devices. Same pin-out (less the 2 outside) as 2.5" drives
- SD card is small, cheap, simple - but bulk connection to computers aren't readily available
- USB flash is more expensive (more interfaces), but scales up well and interfaces are readily available.
- Or some new format - directly inserted onto PCI host cards...
CF may cause IDE/ATA interfaces to re-emerge on motherboards - or PCI IDE-card sales to pick up.