This is my requirements list for putting GlusterFS to production.
First the things I have to setup to have a sane environment.
- All Oracle VM hosts are reinstalled to boot off IDE or USB flash. Performance of that medium may be very bad.
- All Oracle VM hosts are switched to x84_64 arch
- All Oracle VM hosts issue no errors or useless messages in idle service
- All Oracle VM hosts are connected to Cisco SFS3504 Infiniband switch and ib_rdma_cm works between each of them.
- All Oracle VM hosts are connected via Dual GigE (LACP+Vlan) Trunk to the Huawei S5624 switch.
- All Oracle VM hosts can find each other using DNS.
- All Oracle VM hosts have GlusterFS rpms and Fuse installed.
- Nagios is configured to monitor the gluster daemons and volume status.
- Some Oracle VM hosts have local disks, some don’t.
- Local config of the Oracle VM hosts will be minimalized, almost stateless.
- All Oracle VM hosts will log to the Splunk server.
Add more spindles total to 12 or 16. As worst-case estimate that would equal 500-700MB/s throughput.
Build the SSDs into the Oracle VM hosts.