Winning Systems & Security Practitioners 3. Responsiveness

1100 words, 4 12 minutes.


Helmuth von Moltke “No battle plan ever survives contact with the enemy” - Helmuth von Moltke.

This is part 3 of 6 in a short series of posts on winning systems for Information Security practitioners. It aims to plug the gap between policy and products and put you, the practitioner, back in the driving seat. After all if you don’t know what system you’re implementing, how can you possibly decide what products or features are important to you? How can you evaluate what they might be worth?

Congratulations. You’re prepared. You’ve turned the odds around though rigorous application of default-deny. On your LAN, perimeter, desktops, and directory structures within which users store files. What happens when the first enemy incursions are spotted? What happens when an attacker decides to test your defences?

Default-deny isn’t a panacea. As a system it’s good but it’s static, like a wall. Constant probes against this wall can be distracting, particularly against public services, and there’s always the worry that some element of the wall might give way. Unlikely if you truly have default-deny, but it can happen. How can we introduce another system that is adaptive and responsive and further reduces the likelihood of compromise? How can we automatically improve the strength of our security using cues from our environment?

A few simple cues you might recognise from your own network:

  • Repeated failed logins, probably password guessing (1).
  • Repeated connections attempts, probably service scans.
  • Repeated ‘404 URL errors from a web server, probably scans (2).
  • Attempted Email relaying for non-local users, probably spam (3).

(1) You switched to 2-factor years ago, or ssh certificates didn’t you?

(2) You switched to default-deny on your web servers didn’t you?

(3) You already largely outsourced Email hygiene didn’t you?

The responsive system is:

  • Detect early, ideally after just 1 or 2 failures.
  • Block automatically, at the packet filter.
  • Collective punishment is good, why not drop the /32?
  • Propagate the blocked list across systems.
  • Preserve the list across reboots.
  • Manage by exception (unblock something only when asked).

Most commercial network security products, including network and host-based IPS/IDS have some kind of “bad guys” feed. Most also have the ability to dynamically add locally generated IPs to that ban list. Remember rigorously implemented default-deny should already be doing 90% of the work for you. However, for the few services you must have open, and in particular those without strong authentication, these feeds further limit the chances of connection with a bad guy. Personally I find a system of responsive blocking most useful for exposed public hosts/services like this web server. A couple of failed logins or an attempted URL not on the permit list, and the IP address can safely be dropped by the hosts packet filter. Either temporarily or forever depending on the nature of the transgression.

As of now this system has temporarily banned 79 hosts. Bans expire after a week and are imposed after a single failed login.

This is where our analogy with the real world starts to show some strain. In the physical world, activating responses takes effort and requires resources to maintain that effort. In the world of Information Technology we don’t have the same constraints. If we know we are under attack what are we to do? Responsiveness for information security practitioners is more simple than in the physical world. Returning to our wall analogy, as soon as we identify somebody testing the wall, we automatically pour boiling oil over them. That person (IP address) isn’t heard of again. For the same reasons, the 90’s fad for “Internet AlertCons” was mostly pointless. If the Internet is experiencing a wide-scale security threat what are you going to do? Unplug your Firewall? Up your encryption to 256-bits from 128-bits? Temporarily activate 2-factor authentication (when previously you’ve been paying for it but not using it)?

The vast majority of responsive actions should be automatic.

These actions should be tested and primed ready.

Milliseconds count when your defences are being probed.

The best thing about routine responsiveness as a system, is that it gets stronger the longer you operate it. With each probe, with each failure, your list of excluded addresses grows and the amount of noise in your application logs shrinks. More and more failed attempts are relegated to the packet filter where they are computationally inexpensive to quench. We can borrow a term from the philosophy and business publishing for this. Antifragile. It gets tougher the more it’s attacked. Add a honeypot account or system. Give it trivial passwords. Ban anyone who touches it.

This simple, automatic responsiveness can be had for free on most Operating Systems. It should be a part of every standard build, whether or not that Operating System is protected by a network Firewall device already. If it’s automatic and maintenance free, then why not? One day your network Firewall will be misconfigured, or buggy, or fall foul of some magical attack and you’ll be glad of a little host based security. No system is infallible.

In exceptional cases it’s possible to envisage responsive actions which are triggered manually. These actions would be much more significant and resource intensive (and potentially more disruptive) to end users. As a result, it’s important that the impact of them is well understood. This means regular rigorous testing of the procedure. A few examples;

  • Emergency OS update upon next logout (desktops).
  • Emergency package update outside maintenance window.
  • Emergency configuration change outside maintenance window.
  • Emergency storage snapshot.

It goes without saying that individual tasks required to accomplish these actions must be automated into a playbook. They will be triggered outside any maintenance window. Performing them by-hand while under pressure is a recipe for disaster.

If you think responsiveness sounds like a good system for reducing your chances of being hacked, then you’ll love robustness. On the other hand, if you think responsiveness is overrated, and wont do much in practice, then you’ll need some alternative to prevent you getting hacked.

Nick Hutton

Engineer, Investor, Founder, Product Manager

London, England