The Codist

Along with 24 million of my not so closest friends I too got an email telling me that Zappos suffered a breach in which they lost control of their user database through some as yet un-disclosed breach.

Fortunately they did keep the credit card numbers separate from the user accounts and according to them only kept hashed passwords in that database. But losing personally identifiable data still can cause customer pain, and hashed passwords unless done really well can still lead to recovering enough passwords to create trouble. So far there is no discussion of how the exploit happened, or for how long it was active. We may never find out, as the details are usually not something a company wants to discuss.

At least they seem to have owned up to it, which itself is disaster PR 101, although the exact sequence of events probably won't ever be known. But why does this continue to happen?

I've worked for enough companies with sensitive customer information to know that most of them simply don't know, care or allow themselves to worry about what might happen. The problem is generally that customer security is invisible for the most part; it isn't something you can show people, there is no real way to prove it and most customers wouldn't understand the details even if you thought it was a good idea to lay it all out. It only becomes important when it fails. Like a Star Wars missile defense system you can continue to imagine it's working until you see hundreds of missiles in the air.

In most places I've worked people talk about it sometimes but it's rarely as important as a new feature, a prettier UI or some great marketing blitz. Security is what you do to pass some quicky audit. In one financial service company security was worrying about people downloading porn and viruses, even while credit card information remained unencrypted while everyone debated the hardware cost. My healthcare experience featured locked down hardware (no USB devices allowed anywhere) yet the passwords for the production databases and servers was stored in a text file available to half the company. But they passed some kind of audit so everything was cool.

Security is usually not a C-level position. In fact I would imagine a really qualified security executive (let's say someone Bruce Schneier would respect) would probably piss off everyone in the company and find themselves looking for work rather soon. Real security is hard, it's complicated, and it costs money -- and it only gets negative respect. Like a good football lineman your work is best when no one calls your name. If hack attempts and data breaches don't succeed your security team remains invisible. If they do succeed you get fired. What a job...

I do think that following what I know of the NSA isn't done enough, when they build a new system they always have an independent team work that does nothing but try to defeat or break it. If you have sensitive customer information you need to have a team that without inside knowledge tries to hack you. Of course you have to be very careful so it doesn't become a real hack which I am sure scares people off. But you really don't know how secure you are in any area unless you do real world tests, otherwise you are imagining your very own Stars Wars defense system.

Of course you need to consider the most likely vectors, carefully partition data, use as much encryption as is practical, and create enough layers of protection and monitoring to at least confuse or slow down an attacker. The more sensitive information you have the more you need to invest in real security design just like you do everything else. It's not something to add on at the end or worry about it later when something goes wrong.

One of my favorite security quotes came at the healthcare company when I reported that the production passwords were available to virtually everyone in the company to the CIO he said "Oh we trust our employees".

Something to remember before you write 24 million depressing emails to your customers: security is not about the 99 things you did but the one thing you didn't.