thattommyhall.com

When I first heard about ZFS and its features, I was intrigued by a comment by Bill More about the possibility of having a database or other app directly consume the DMU that ZFS uses for filesystems or volumes. After I did a spot of research when editing the ZFS page on Wikipedia I noticed the “Last Word In Filesystems” pdf has been updated since I last looked, here are the 2 pages that excited me. With Suns recent acquisition of MySQL and lustre we seem to have arrived there now. Lustre support is excellent as it solves the only failing of ZFS, that it is not clustered.

The in-kernel CIFS stuff gets a mention too

Great work  

See the full presentation and Bill Moore’s slightly outdated video

Posted by tom under ZFS | No Comments »

After me banging on for a year about how cool ZFS is, my boss is finally convinced and wants to get a Thumper (or 2). Depending on budgetary constraints we may be getting a couple of these to implement a warm backup solution for our current data and about 5 years worth from now (they are 24Tb each)

48 Drives in a 4U rackmount for 20k, great value and unbeatable storage density.

Posted by tom under hardware & ZFS | No Comments »

Exporting a ZFS filesystem via NFS is easy, you just use the command

zfs set sharesmb=on POOLNAME/FSNAME

In order to share via CIFS (sometimes called SMB) to windows clients, you used to have to use Samba. This is not a problem for simple setups that dont use Active Directory, but can be a pain if you do as the ACL model in Windows is completely different to Unix (One of the few areas Windows is better in my opinion). Samba get around this by adding hooking up to OpenLDAP, adding a VFS layer (pdf) and reimplementing Kerboros/LDAP to match the Microsoft version, all of which are far from trivial to set up. I love Samba and think it is one of the best pieces of free software there is.

Sun have just released a CIFS implementation in the SunOS kernel, based on a stack that they got when they bought Procom. As well as serving up files via CIFS, they have modified the ACL system to be a sort of hybrid or old style Posix and Windows style ones. I think they have been planning this for a while as NFSv4 ACLs are similar to NTFS ones but still map on to Posix style eventually. So the end result is you will be able to write

zfs set sharecifs=on POOLNAME/FSNAME

and have the filesystem exported. The fact that it is in-kernel and integrated with ZFS is cool, but this on-disk storing of the ACL information is the best feature. Check out Mike Shapiro and Alan Wright’s blogs on the topic, the full spec and the project page for more details.

That is not to say Samba is useless, I was very excited by the examples here, particularly the patch to implement shadow copies using ZFS, but maybe people will move away from it in favour of a more tightly integrated in-kernel implementation. The CIFS client in Linux is now in-kernel (still written by the Samba guys though) as it is in SunOS and Darwin (Sun ported the Darwin version). Untill NFSv4, Samba was definitely the best way to share files between Unixes, due to some nifty Posix Extensions and the throughput from the efficient implementation (beating in-kernel NFS from Linux in my testing, it may still be the best way). Samba beats Windows servers for throughput and number of clients served, the SMBTorture suite they developed even got mentioned (search for Evil Empire) by a Microsoft guy as they now use it to test their implementation.

When I get it on my NAS, I’ll post benchmarks.

Posted by tom under Samba & ZFS & OpenSolaris | No Comments »

No silent data corruption because of a checksum on each block, copy on write means no fsck EVER (it does not even have one), no RAID5 write hole, add disks to the pool on the fly, put volumes there as well as filesystems and export them via iSCSI or NFS, quotas, compression and it is easy to use. Check it out here, or watch this video

It is so easy to use, here is my command line session when adding 4 disks to my array. Took me perhaps 5 minutes and its worth noting that there was no pause between issuing each command as its all pretty much instant. I had to import the pool at the beginning as I had just installed OpenSolaris developer edition to see if it was any better than Nexenta (it was not). The spacing of the output is a little off, but hopefully its still understandable.

# zfs list
no datasets available
# zpool import Toms
cannot import ‘Toms’: pool may be in use from other system
use ‘-f’ to import anyway
# zpool import -f Toms
# zfs list
NAME USED AVAIL REFER MOUNTPOINT
Toms 1.33T 9.84G 40.4K none
Toms/500G 723G 9.84G 723G /share/500G
Toms/Notes 25.9G 9.84G 25.9G /share/Notes
Toms/files 25.6G 9.84G 25.6G /share/files
Toms/share 532G 9.84G 46.4K /share
Toms/share/SetupFiles 72.2G 9.84G 72.2G /share/SetupFiles
Toms/share/TalkingBooks 139G 9.84G 139G /share/TalkingBooks
Toms/share/Tunes 57.5G 9.84G 57.5G /share/Tunes
Toms/share/Videos 263G 9.84G 263G /share/Videos
Toms/share/apps 159M 9.84G 159M /share/apps
Toms/share/downloads 26.9K 9.84G 26.9K /share/downloads
Toms/xbox 50.7G 9.84G 50.7G /share/xbox
# zpool status
pool: Toms
state: ONLINE
status: The pool is formatted using an older on-disk format. The pool can
still be used, but some features are unavailable.
action: Upgrade the pool using ‘zpool upgrade’. Once this is done, the
pool will no longer be accessible on older software versions.
scrub: none requested
config:

NAME STATE READ WRITE CKSUM
Toms ONLINE 0 0 0
raidz1 ONLINE 0 0 0
c2t3d0 ONLINE 0 0 0
c2t0d0 ONLINE 0 0 0
c2t2d0 ONLINE 0 0 0
c2t1d0 ONLINE 0 0 0

errors: No known data errors

# zpool add Toms raidz c2t4d0 c2t5d0 c2t6d0 c2t7d0
invalid vdev specification
use ‘-f’ to override the following errors:
/dev/dsk/c2t4d0s0 is part of exported or potentially active ZFS pool
mypool. Please see zpool(1M).
# zpool add -f Toms raidz c2t4d0 c2t5d0 c2t6d0 c2t7d0
# zpool status
pool: Toms
state: ONLINE
status: The pool is formatted using an older on-disk format. The pool can
still be used, but some features are unavailable.
action: Upgrade the pool using ‘zpool upgrade’. Once this is done, the
pool will no longer be accessible on older software versions.
scrub: none requested
config:

NAME STATE READ WRITE CKSUM
Toms ONLINE 0 0 0
raidz1 ONLINE 0 0 0
c2t3d0 ONLINE 0 0 0
c2t0d0 ONLINE 0 0 0
c2t2d0 ONLINE 0 0 0
c2t1d0 ONLINE 0 0 0
raidz1 ONLINE 0 0 0
c2t4d0 ONLINE 0 0 0
c2t5d0 ONLINE 0 0 0
c2t6d0 ONLINE 0 0 0
c2t7d0 ONLINE 0 0 0

errors: No known data errors
# zfs list
NAME USED AVAIL REFER MOUNTPOINT
Toms 1.33T 888G 40.4K none
Toms/500G 723G 888G 723G /share/500G
Toms/Notes 25.9G 888G 25.9G /share/Notes
Toms/files 25.6G 888G 25.6G /share/files
Toms/share 532G 888G 46.4K /share
Toms/share/SetupFiles 72.2G 888G 72.2G /share/SetupFiles
Toms/share/TalkingBooks 139G 888G 139G /share/TalkingBooks
Toms/share/Tunes 57.5G 888G 57.5G /share/Tunes
Toms/share/Videos 263G 888G 263G /share/Videos
Toms/share/apps 159M 888G 159M /share/apps
Toms/share/downloads 26.9K 888G 26.9K /share/downloads
Toms/xbox 50.7G 888G 50.7G /share/xbox
# zpool upgrade Toms
This system is currently running ZFS pool version 8.

Successfully upgraded ‘Toms’ from version 7 to version 8

Posted by tom under Nexenta & ZFS & OpenSolaris | No Comments »

I love the SunOS kernel. ZFS, Dtrace, Zones all make it desirable. But OpenSolaris is a code repository, not a distribution and you have to go elsewhere if you want a free distro. Nexenta looked the most promising, but then went quiet for a while (at least on their webpage) after shifting their focus to a smaller package set they called Nexenta Core Platform. I was very pleased to see an offer of collaboration from the Debian project leader in their forums and now Nexenta.com has released a new NAS package (not free though) and at the same time Suns “Project Nevada” seems to be bearing fruit after hiring Ian Murdock (the chap who started Debian)

OpenSolaris Developer/Community editions pale in comparision to Ubuntu for range of packages and usability, perhaps Nevada can sort it out. I am hopeful that Nexenta get it together and be the GNU/Solaris distro I really want (see here for some screencasts by Martin Man showing how cool it already is).

Posted by tom under Nexenta & ZFS & Ubuntu & OpenSolaris | No Comments »