Part one: Storage migration, disaster recovery and friends

This is the first post of a series describing recent changes I did, some data loss, recovering from it and evaluating damage.
All posts:


Starting point.

I am building a new Xen Host for my home lab. It was supposed to handle one or two full Ceph labs at high load.The old machine just couldn’t do that.


What I had was a Core2 Q6600 quadcore CPU on an Intel S3210 board (IPMI, yay). It had 8GB of Ram, a IBM M5015 Raid Controller and Dual Nics. For storage I had a Raid10 over 4x2TB WD Green drives fronted by a Raid0 Flashcache Device build from two Samsung 830’s. Due to the old chipset the SSDs were limited somewhere around 730MB/s read/write speed.

The main problems were lack of CPU instructions (nested paging etc) for advanced or bleeding edge Xen features.

  • Memory overcommit using XenPaging only works if you have a more recent CPU than mine. (Of course this defeats the point since a more recent Xeon can handle enough RAM in the first place. But still)
  • The second thing was that PVH mode for FreeBSD needed a more recent CPU and last,
  • Nested Virt with Xen is getting somewhere which would be interesting for running ESXi or many Cloudweavers instances w/o performance impact

So, I couldn’t have many nice things!

Also I knew the consumer SSDs had too much latency for a highspeed cache.

For Ceph there was the added requirement of handling the Ceph Journals (SSD) for multiple OSDs and not exposing bottlenecks and IO variances from using the same SSD a dozen times.


I’m unhappy to replace the server while it was so far never really over 2-3% of average CPU – but since I want to do A LOT more with Ceph and Cloudweavers it was time to take a step forward. I spend some time calculating how far the step could be and  found that I would have to settle somewhere around ~1600 Euro for everything.


2 thoughts on “Part one: Storage migration, disaster recovery and friends

  1. Pingback: Part three: Storage migration, disaster recovery and friends |

  2. Pingback: Part two: Storage migration, disaster recovery and friends |

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s