Your SIEM Is Worthless (And It's Your Fault)

Excessive firewall denies. Multiple login failures. Excessive firewall accepts. Outbound communication to known malicious sites. You’ve probably got these rules on your SIEM. You’re getting alerts from them and you’re spending your time ruling out false positives and updating the rule when something in your environment changes and you’re happy knowing that if someone tries breaking into your systems, you’re going to catch them trying to guess the domain admin account. You’re going to catch them moving laterally (and failing to move). Attackers stumble around, they make noise that normal users don’t make. Right? Right?

How many offenses do you have in your SIEM? How many alerts? How many emails do you get? And how many of them do you ignore? How long is your “run book”? How long do you spend telling new employees what alerts represent “acceptable risk” or “no further action”? Because I guarantee it’s too long. Too many. Too much. Because those rules are crap.

Let’s talk about maturity levels. I’m a security consultant and we are just as bad as anyone else at this. Far too often we recommend for low-maturity environments to use their SIEM to find policy violations. We might not consciously understand this, but that’s what we’re doing. Any time we write a rule that shows telnet connections, that shows communication to sites on a blacklist, that tattles when a user is sending more than 500mb out of the company’s network, we are codifying policy violations into a security tool. And that’s fine, right? Those are security issues!

But they’re not.

But the security team needs to know about it!

No we don’t. Those are policy violations. If someone is breaking policy, their manager needs to know, not the security team. And transferring a huge file out of the company might be an indicator of compromise, but most often it’s just someone uploading a pirated copy of Zootopia to their Dropbox account. Policy violation. HR issue. Not a security issue. So why does security get alerts on that? Is your million-dollar tool there to tattle on your users or to find security incidents? You want to talk about “next level of maturity”, let’s skip “policy violation” and go straight to a step I like to call actual value.

When’s the last time you responded to a SIEM alert and it was actually a breach? Was actually a security incident? It’s probably been a long time, if ever. So what value are you getting out of your SIEM?

Let’s start with excessive firewall denies and multiple login failures and other serious nonsense like that. They’re seriously nonsense. No one cares. It’s not an indicator of anything but poorly configured systems, which is a really terrible use of the resources of a security department. Why does it take two, three, four smart, trained, full time people to run a SIEM? Because most of it is hunting down non-issues. Most of it is tuning and busy work. Very little of it is responding to security incidents.

Turn.

Those.

Rules.

Off.

Simple as that. They are not indicators of compromise. They are not security incidents. Just do it. Turn them off. They are worthless. They are worse than worthless; they are actively harmful because it uses up the resources of your security team. They hide the actual incidents behind trivial nonsense and force you to spend your days hunting down what service account had its password changed but never updated on one system. If no one has noticed yet, no one cares. Move on. It’s not a security incident. It’s not. You know it’s not.

So what rules should we turn on? You’re not going to like the answer. Because those rules won’t fire. They really won’t. And that’s a good thing! Your SIEM shouldn’t fire! Ever! If you get an offense or an alert or an email from your SIEM you should be screaming! It should keep you up at night! If you’re getting alerts that you skim and move past, your SIEM is working against you, not for you. Turn that noise off. How many alerts per week should you have? NONE.

Instead of multiple login failures, why not a honey account? An account with a tantalizing name like “domain admin” with privileges to nothing? As soon as that account logs in anywhere, you hit the red button and start the klaxon. Because the only person who would ever use that account is an attacker inside your network. If the honey account even fails to log in, even once, that is an indicator of compromise.

Instead of multiple login failures, why not successful logins from different geographical regions? You don’t have to worry about people failing to log in. If someone fails to log in, there is quite literally no compromise. It’s only a compromise if someone succeeds. And then, only if they actually do something bad. Never ever ever is a mere login (or login failure) an IoC. But one account logging in from the US and Russia at the same time? One account connected from the UK and China at the same time? Best case scenario it’s a credential sharing policy violation. Most likely this will never fire unless an account has been stolen.

User profiling is an important tool too. If you don’t have a user profiling tool, get one. Get one that integrates into your SIEM and can fire an alert when someone does something they shouldn’t do. Because multiple login failures is absolutely worthless. But if Bob from accounting is logging into the machine of Alice from sales… THAT is an IoC. That’s a problem.

How about clearing event logs? Every system worth its salt will create an event when its event logs are cleared. Key your rules off that because what’s the first thing an attacker will do to hide their tracks. They will clear the event log that can tattle on them. Many attackers know this and won’t do it, but it’s worth looking for. Because this one will never fire unless you’ve been breached. Or you have lazy admins. But that’s a policy violation right? Make their manager shut that behavior down.

Instead of excessive firewall denies, how about trusting your IDPS? Yes. You should do this. You should log that to your SIEM, but you should also turn off the rule that turns those IDPS alerts into SIEM alerts unless your IDPS only alerts on successful breaches. Too often IDPS alerts on blocked attacks! Why?! You don’t need to tell me you’re doing your job! Shut that noise off! Alert on successful breaches. Yes, your SIEM will go quiet for a long time. Hopefully years. And that’s okay!

So what should the security team do in the meantime? You’re already writing corporate policy. That’s great. You’re reviewing your firewall policies on a regular basis, right? Making sure they make sense and are locked down? If not, do that. You’re reviewing your proxy rules right? All of those whitelist exceptions are still needed? You’re doing formal risk analysis of changes to your environment? Do that. Develop a risk analysis framework, or better yet adopt one. SANS has one. Use it. You’re phishing your employees, right? Do that. As a follow up, you’re formalizing end-user education? Making sure your users are working for you, not against you? Seriously, DO THAT. You’re running vulnerability scans on your entire network? Do that. You’re setting up application whitelisting? Do that! You have database activity monitoring (again with proper rules and no stupid babysitting)? You’re running DR tests and security incident drills and purple-teaming? All of the things you’ve said “I’d love to do that but I have to babysit the SIEM”? Stop babysitting that SIEM! Do your job!

Your manager wants to see value from the SIEM. Implementation was a long process, they’re spending a lot of money for employees to manage it, and let’s be honest, SIEMs are expensive. But the security team has too much real work to do to sit there responding to false positives. Your rules should only alert on actual security incidents. If they’re alerting on anything else, turn those rules off and figure out rules that make sense. Want to show value to your manager? Print off a weekly report of firewall denies and have it sent to your networking team. Send your IAM team a copy of the login failures. You never need to see those reports, you never need an alert, and it doesn’t matter if anyone ever takes action on them. That’s not your responsibility. Your only responsibility is the security of your company, not the behavior of your users.

Stop being a slave to your SIEM and actually get to work doing what you were hired to do: secure your damn company!