The Art of Systematic Security Assurance (Part 1): Redefining Cybersecurity Investments

In your career, you gradually realise that scanning your way into security doesn’t work. Let’s say you test an application, you retest it, and the following year you go and test the same application, only to find the same bugs. You end up wondering why the team isn’t fixing anything and understanding you’re not adding as much value as you should – and this goes for all third-party pen testing companies.

Troy was my mentor without even knowing it. I learned more in six months of working with Troy than in the previous ten years. Why? Troy has created a method that is easy for anyone to comprehend, regardless of background. It seemed like the work of a dozen people, done over many months – not just one person. It was a much better approach than just doing a yearly audit. I believe it’s how everyone should be approaching security.

The systematic, scalable approach I’m proposing is the work of many years, but not my years. I’m standing on the shoulders of giants, of the people I’ve worked with. I’ve taken little bits and pieces from all my experiences.

My fundamental belief is that applications or assets are more than the sum of their parts. There’s an application and some source code. Running a pen test tells me something about it. Some vulnerabilities get uncovered. But we must also consider how the application is built, its architecture, and its data. And then there’s the business perspective on that application.

Tracking and creating relationships between all these layers can help make better decisions.

Let’s say I have a medium-priority result from a pen test. Knowing the architecture of my application, I understand it enough to know that what the test has uncovered is only executable behind a firewall, and another setting mitigates it. Moreover, the data that can be accessed is unimportant. Given all this, the initial result is not that important to me. But you need a systematic approach to come to that understanding and conclusion.

By the time I come to the point of pen testing, I already know many things about the application. It’s not a black box for me. And the fact that that’s not everyone’s experience was shocking to me. My question is: what are you spending your money on? What results are you getting?

In the end, this kind of systematic approach results in risk reduction, automation, and scale.

We talk a lot in DevSecOps about reducing friction between the dev and security teams. Troy’s approach is about asking relevant questions and getting answers from the right people. You find business, security, dev related questions all in one place. If there’s something we don’t know, we start a dialogue. Having one single source of truth starts reducing and removing friction, internally at least.

This information is always missing when a company asks a third party to do a pen test. The client needs a pen test to check a box, and that’s it. But this doesn’t work well. We regularly see so many companies get compromised, despite having internal security and pen testing. Why? Because they lack transparency. It’s the reason Chaleit proposes a new holistic approach to pen testing.

Earlier, I was pretty happy with clients saying they wanted a white box and a code-assisted pen test. I rolled with it. But then I started thinking about the whole and looking at architecture diagrams and threat modelling.

Maybe I report something as a critical priority based on the information I have – or, better said, that I don’t have. Reporting something without context instantly creates friction. One mistake and the team on the client side throws the report away. They think, “This person doesn’t know anything about my application. I’m not going to take any of their suggestions. I don’t need to fix anything”.

Pen testers must know that asking questions and getting more context is okay. Otherwise, we’re just trying to prove ourselves, and our only goal becomes finding multiple reds and yellows. But it’s not about proving ourselves. It’s about increasing overall security.

Pen testing is still relevant and necessary to unravel a lot of things. But we realised we must widen the scope to complete an application’s security assessment.

Let’s be honest about security. It’s not trivial to look at a system, break it down into the components, break those components down into things that might be vulnerable, and model and map them the same way you would model and map a city. They’re complex systems with several different layers. You need specialists to deal with that.

The security ecosystem has many middle specialisations: vulnerability management, AV endpoints, incident management, forensics, architecture, etc.

Tying all this together is where security is not quite at the same level as airlines, for example. We’re not yet at the level where we are doing holistic life-cycle analysis about safety in the cyber world in the same way we do it in the physical world. You understand and consider multiple contingencies based on that particular thing’s risks, threats, and vulnerabilities, like an aeroplane and the likelihood of something happening. We need to think about that life cycle more maturely in cybersecurity too.

Curious to find out more? Read Troy’s Introduction to Madness: a path to systematic security assurance.

The conversation continues, so stay tuned! This blog post is part one of a three-part series in which Ankit and Troy unpack all it takes to build a systematic, scalable approach to cyber security.