Dominic Cleal's Blog

Sat Jan 2 20:53:49 GMT 2010

permalink BBS2 for a home NAS, part 5: OpenSolaris to FreeBSD

While updating my blog earlier, I noticed a comment stuck in the moderation queue of another article that I hadn't seen until now. Apologies to Steven who has probably long since given up hope of a reply, but here it is anyway.

Sorry to hijack this comment, but I was wondering what the state of your quest for zfs on the BBS2 is? I'm considering buying one myself, for the exact same setup, OpenSolaris + zfs (the alternative is a more traditional NAS like the QNAP 439). I'm especially interested in the performance of the system (as a NAS), and whether you made any progress on the problem with the network card drivers. Have you tried/considered using Nexenta iso OpenSolaris?

thanks!

Steven

I mentioned earlier last year that a kernel engineer was looking into my nasty Realtek RTL8111/8168B issues under OpenSolaris on the BBS2, but I didn't provide an update.

At the time, a kind engineer at Sun had seen this post and passed my contact details onto somebody who could help. For a number of weeks, I set up the BBS2 with a serial console via another box and a small GigE network that the Sun developer could use to try and trigger the bug between the two machines.

This was fairly successful and after a little time, he had managed to get the delay for a network restart down to about 20-30 seconds with modifications to the rge kernel module and was confident that it could be reduced to just seconds. I had on a couple of occassions seen complete network dropouts where the kernel didn't seem to notice that the chipset had died and so didn't reset it. Unfortunately, I couldn't reproduce these at all and so the developer couldn't provide any more help.

Since that fizzled out in April/May, I hope the fixes that Sun came up with have made their way into the OpenSolaris source tree, but I haven't checked the change logs recently. Thanks to both Sébastien and Winson for their efforts!

After leaving the BBS2 for a few months, I decided to install FreeBSD 7.2 after replacing my desktop in October. At first I ran into a few issues with lockups while trying to copy data on, but with some tweaks, including recompiling the kernel to increase the possible kernel address space the system has been pretty much perfectly stable since.

I'll be upgrading the box to FreeBSD 8.0 soon in the understanding that it has improved memory handling for ZFS and won't need the tweaking and recompiling that was necessary to run ZFS under 7.x. FreeBSD is definitely recommended if you're looking for ZFS, though it isn't for the fearless!

5 comments
Archives


Comments On This Entry

Seth
Thanks for the updates. I'll be installing OSol on my brand-new BBS2 that I ordered yesterday. I'm not sure I'll get used to Solaris quickly, so I ordered the BBS2 ahead of time (before my koolu linux server-let is end-of-lifed or takes a dump :)). This way, I can transition the applications gradually (if at all) but I intend to move all storage to ZFS/iScsi/NFS right away. I'm running a zfs-fuse setup at the moment and frankly the performance isn't enough (although it's getting a lot better with 0.6.0) I'll prolly let you know how I fare

Dominic
Good luck Seth. I first came across Solaris at work, but it isn't really that different to Linux or other Unixes. The Indiana builds (OpenSolaris 2009.06 etc) have the GNU tools further up the path and are even more similar. The administration tools in Solaris are top notch though, such as SMF (service management) and are really nice to work with. The documentation at docs.sun.com is usually very comprehensive too. The hardest part is that the software repositories are very lacking, unless you use Blastwave or Sunfreeware.com (or Nexenta?).

Seth
Hey, I found this interesting link on rge drivers: At the time of the composition, Solaris Express Community Edition was at build 98, and the rge driver in that build still has a small problem with the RealTek card on this board. In order to enable it, you need to add the following line to /etc/system: set ip:dohwcksum=0 (http://blogs.sun.com/pfuetz/entry/eco_responsible_small_home_server) I'm hoping this will do some magic trick to stop the network from dropping (?) I'll have to read up more of course Sadly, I did play around with OpenSolaris before, but I managed to break the pacakage manager by using packages from blastwave. So that system got unusable. (Also, for some reason I toasted my 'virgin' BE so I could'nt actually go back in time). I'll be careful this time of course, and ZFS is a big motivation

Dominic
Hi Seth. Yes, the dohwcksum fix is documented also by Daz (http://sigtar.com/2009/02/12/opensolaris-rtl81118168b-issues/) who tried a number of different fixes for the same motherboard but eventually resorted to using an Intel card instead (which isn't possible in the BBS2). Cheers.

iMx
Hey, the latest builds of Opensolaris have an updated rge driver, ive been using b132 recently - i think the fix was committed in b127 or there abouts; b133 has a bug in the sharesmb zfs set command, in that it doesnt work ;) Also, the gani driver works pretty well for me... Ive been having some issues with IO stalls, when writing over the network it runs at 28-32MB/s then stalls for a few seconds - a bios upgrade greatly improved this, even though i had to install windows to do it! Interested to know what stats you get on Opensolaris or FBSD with zfs, ive been tuning everything from ARC cache, to write limit override - nothing seems to fix it! Works flawlessly with Openfiler in raid5, but i dont want to use that!

Comments for this entry are now closed.