Requirements for judging KVM

Because there is still a lot of confusion re KVM vs Xen

I made a self-evaluation check-list for finding out if you’re qualified to judge if something is better / more suited for larger-scale virtualization than KVM:

Option a) You have used and/or built environments based on:

[x] FreeBSD jails
[x] Solaris Zones
[x] AIX LPAR
[x] HP-UX vPar
[x] VMWare esx4i
[x] VMWare esx5i
[x] Virtualbox (even with native iSCSI)
[x] Xen 2
[x] Xen 3
[x] Xen 4
[x] Citrix XenServer 3
[x] Citrix XenServer 4
[x] Citrix XenServer 5
[x] Citrix XenServer 6
[x] Oracle VM 2
[x] Oracle VM 3
[x] KVM

Overall outcome:

Not qualified

Option b) You like KVM

[x] nothing more

Overall outcome:

Qualified

Devops reality

“Right now every developer has access to all the passwords in the environment. The only line of defense is how convoluted the setup is so finding anything is terrible chore”

Monitoring software updates on CentOS – but not security updates

You might already know the yum plugins – there is a plugin that enables “downloadonly” fetching of patches before the update, or another prominent one that checks all updates whether they are security related.

Some people even have built cool Nagios checks for report the updates[*], but none of them seem to actually check if yum really reports a security update as a security update. Since I had been testing most of those plugins I had been really wary about the results. For example last year there was almost half a year delay in the CentOS updates, surely there would have to be a security update in those. But none were reported. Over the last weeks there was a few critical updates on Linux, giving a good time to test – again. The result was…

For the last time guys:

Using the yum security plugin and CentOS DO NOT COMPUTE!

The CentOS team is not adding the CVE data the plugin wants to read. Yes it will tell you “no security updates” but it is WRONG, the plugin does not handle missing security data and gives false results. GET it in your heads.

False sense of safety:

Here you can see it reporting no security updates:

-bash-3.2# yum –security check-update
Loaded plugins: downloadonly, fastestmirror, security
Loading mirror speeds from cached hostfile
* base: centos.kiewel-online.ch
* epel: ftp.uni-koeln.de
* extras: centos.kiewel-online.ch
* updates: centos.kiewel-online.ch
Limiting package lists to security relevant ones
No packages needed, for security, 10 available

Under the hood:

Adding -v we get to see the actual packages it was checking:

up:condense time: 0.000
updates time: 10.740
–> 2:libpng-1.2.10-17.el5_8.x86_64 from updates excluded (non-security)
–> openssl-devel-0.9.8e-22.el5_8.3.i386 from updates excluded (non-security)
–> openssl-0.9.8e-22.el5_8.3.x86_64 from updates excluded (non-security)
–> openssl-devel-0.9.8e-22.el5_8.3.x86_64 from updates excluded (non-security)
–> glibc-2.5-81.el5_8.2.i686 from updates excluded (non-security)
–> glibc-devel-2.5-81.el5_8.2.x86_64 from updates excluded (non-security)
–> glibc-2.5-81.el5_8.2.x86_64 from updates excluded (non-security)
–> glibc-headers-2.5-81.el5_8.2.x86_64 from updates excluded (non-security)
–> openssl-0.9.8e-22.el5_8.3.i686 from updates excluded (non-security)
–> glibc-common-2.5-81.el5_8.2.x86_64 from updates excluded (non-security)
No packages needed, for security, 10 available

Now look at that list…

LibPNG? A security fix

OpenSSL? A security fix

Summary:

Please, don’t rely on yum security on CentOS, either manually verify your updates or chose a distro where the repo data is “spiced” with the security update data and all packages are correctly referencing the CVE Ids they’re fixing.

Also, afaik none of these RH-based distros will warn you based on a security hole alone. They’ll just warn you once there is a fix for an issue.

[*] A very good howto is at  http://flakrat.blogspot.de/2012/01/how-to-configure-nagios-checkmk-to.html

Citrix XenServer and XAPI

Let me give them a chance to express their coding abilities in their very own words, which I found while working on a filter that supresses “chatty” log output from XenServer.

[2012...|debug|xs1|14262 inet-RPC|dispatch:logout D:...|dispatcher] Unknown rpc “logout”

Yes, this piece of shit doesnt even understand it’s own RPCs.

Nagios notification tuning

Woah…. I switched my nagios and notification config to the new distributed WATO. It’s really nice to have a web frontend for user / mail configuration. There’s also an editor frontend for the mail body/subject. Remains to be seen how long it’ll be until customers come around with SMS  and escalation requests

This is a snippet from the most interesting part:

The downside is that I had to go through the whole service threshold and notification tuning again. I already had the configs centralized in a mercurial repository, and now everything has to move into the WATO config.

I had like 8 unacknowledged services and ~20 nagios restarts for config updates. All together that gave me just somewhere around 100 mails…

And the most awesome thing of the test:
Rolling back the WATO config using the last snapshot once I noticed an error:

 

Oh, and *psssst* for those who run Nagios clusters in multiple datacenters and didn’t yet find time to test and play with the latest GIT versions:

Don’t miss where it says peer-to-peer replication…

Official rules for SSL notaries

This is a nice reading: New, official rules for running a SSL notary have been released by their forum.
http://www.cabforum.org/Baseline_Requirements_V1.pdf

Keep an eye on how high some of the requirements are, i.e. always need two persons to access the root CA key, and never use it to sign client certificates. And *then* notice how they often sneaked in a “SHALL” instead of a “MUST” where it would cost the registrar actual money to implement better processes.

Dreamhack 2011 – 120gbps upstream

Some very great pictures from the last dreamhack, along with some awesome numbers.
On the other hand that makes me miss the good old cosy 300-people Megabit even more.
 

 

 

http://www.extremetech.com/extreme/107245-inside-the-worlds-largest-lan-party?print via Webhosting talk

local suid in linux ;)

There’s a ugly local suid root in linux that is only patched since, well, very recently. What’s worse there’s also a number of exploits already because this has apparently interested many people.

Further reading:

http://blog.zx2c4.com/749

Otherwise… Well. If you can, search (find -local -perm 4000 / ) and disable all your suid binaries (i.e. start X as root via login manager, then you don’t need suid startx) It’ll be over at some point lol.

The FreeBSD user in me has a strong urge to let out some laughter…

kldunload -f linux.ko

Troubleshooting for lm-sensors (early morning edition)

Suppose you want to add the lm_sensors plugin from the check_mk exchange on your desktop PC. And it just won’t work.

Problem determination guide:

Is lm-sensors installed including the support libraries?

If yes:

Has sensors-detect been run?

If yes:

Does it detect your PhenomX6?

If no:

Does acpi -t correctly print temperature?

If no:

Can you see the CPU thermal stuff in proc??

If no:

Did you book a new VPS at edis.at yesterday?

If yes:

Compare these pictures to identify the system you’re working on:

The new vps —>>           

The overclocked PC  —>>

 VPS != PC

Followup on “on-demand provisioning is stupid”

I learned about the technique Mise en place via my girlfriend who thought she needs to professionalize her weekly baking endeavours.

Read http://en.wikipedia.org/wiki/Mise_en_place and mentally apply this to system provisioning in a large (automated) IT.

This is good followup to my old article http://deranfangvomende.wordpress.com/2011/03/31/are-you-really-done-virtualizing-or-did-you-stop-at-the-start/ where I tried to explain why it is a big waste of time to install each server when required, cut each new SAN lun for the server, assign zoning, provision each VLAN when it’s needed instead of having it ready on time.