Someone (on #ceph) is preparing patches for Ceph to successfully build on FreeBSD!
He sounds dedicated and knowledgeable enough to make it work.
While chatting he said he’d try to plug it into the GEOM block layer framework and also add some ZFS spice.
On linux you have /dev/rbd for blocklayer access (“RADOS block device”) and use LVM (and/or md) to architect more layers of storage on top of this, but on FreeBSD you can really just tuck it into GEOM and build anything you want. GEOM is “*one* tool for all” written by *one* guy, so it’s a lot less messy than mixing all kinds of different stuff like LVM & multipath & ecryptfs & EFI & MDraid & ionice to get to the same goal. (Of course there also has been a usual share of bugs made worse by the fact that there’s less users and less devs to find / fix the bugs. But the design is
heavily leaning on Veritas far better making the non-broken things a charm to work with)
Now, to come back to Ceph on FreeBSD… you’ll have a production-stable & tested ZFS version, a advanced block layer via GEOM and the most modern object storage/filesystem mix Ceph in one box. If you mentally add in HAMMER then *BSD is finally getting to one of the top positions when it comes to the most modern filesystems.
I guess in 1-2 years time there will be a lot of performance tuning work for the FreeBSD Core team 😉