Penetration testing has long been a staple of cyber security programs. Yet too often, organisations run tests that look good on paper but do little to strengthen their actual defences.
Breaches continue to happen in companies with all the right certifications, regular tests, and stacks of tools. The problem is what happens after testing. As part of our Penetration Testing Decoded series, we sat down with Shawn Thompson, Founder and Managing Director of Praexian Security Alliance, an experienced CISO and former government red teamer, to better understand the challenges organisations face and how they can focus on what really matters.
Commodity pen testing is no longer useful
Standardised, checklist-driven tests can be automated away. They identify issues, but rarely lead to meaningful change.
“It’s easier to find things than it is to actually fix them. With today’s volume of vulnerabilities, finding problems has never been easier but remediation is where the real effort lies,” Shawn agrees.
This creates an industry paradox: Pen tests keep getting commissioned because compliance demands them, but many organisations still struggle with breaches that bypass or outlast those same tests.
Shawn recalled his early days working in federal government security: “We were brilliant at breaking things. Every system we touched, we could compromise. But after a while, I moved to the remediation side — and it was much harder. I spent two years working on fixes, and it dwarfed the effort of attacking.”
That imbalance remains. Vulnerabilities are counted in the millions. The real challenge is prioritising what to fix and how to embed problem-solving into the organisation.
Rethinking what a pen test should deliver
So, what should modern penetration testing achieve? Not just a report, but clarity on the following:
Crown jewels of the business (systems and processes that carry real financial or operational value)
Pathways that attackers would exploit to reach them
Real-world feasibility of exploitation, not just theoretical vulnerabilities
Actions needed to close gaps
Shawn described how effective tests used to start with a whiteboard: “We’d decompose the architecture, understand the business strategy, and figure out how the adversary would actually try to get in. It wasn’t just testing systems, it was solving a problem creatively.”
Why SOCs don’t save you
Pen testing can’t be separated from detection and response. A striking example came from a recent engagement where Chaleit gained domain-level access in three hours. The client’s SOC — well-funded and staffed — failed to detect it.
As Shawn explains, “Most SOCs end up grinding through low-value alerts. Analysts are exhausted, distracted, and miss the signal that matters. It’s a broken model if you’re relying on that as your parachute.”
This means pen testing reports must not only highlight vulnerabilities but test whether defences like the SOC actually respond when attackers act.
Building a smarter pen testing model
Not every “critical” vulnerability is equally urgent. In practice, less than 2% of scanner-labelled criticals truly matter. The distinction lies in context.
Shawn’s philosophy is built on a systematic synthesis of three critical contexts: the Business Context (what are our crown jewels?), the Technology Context (where does the vulnerability live?), and the Threat Context (is a relevant adversary actively exploiting this?).
This synthesis is the engine for what Shawn calls "Fierce Prioritisation" — a data-driven discipline for identifying the true emergencies.
Chaleit are aligned — we often reframe these as emergencies during our client engagements. That shift in language helps CISOs and executives focus attention where it counts, instead of spreading themselves thin across a sea of “critical” findings.
Pen testing today must be:
Context-driven – Synthesising business, technology, and threat intelligence to find what’s truly critical.
Prioritised – Fiercely focused on the handful of risks that pose a material threat.
Problem-solving – Aimed at measurably reducing risk, not just ticking compliance boxes.
Iterative – Run in cycles to test improvements, not as one-off annual events.
“If you understand your business, your technology stack, and the threat actors you actually care about, you can build a defensible position. That’s where pen testing still adds real value,” Shawn sums it up.
Key takeaways
If your pen tests leave you with long reports but little change, it’s time to rethink the way you approach the process. Remember these ideas before you schedule yet another yearly pen test:
Finding is easy, fixing is hard. True value comes from remediation, not reports.
Commodity testing is obsolete. A synthesis of contexts must guide testing priorities.
SOC failures are common. Pen tests should validate detection, not just prevention.
Embrace "Fierce Prioritisation." Focus on the true emergencies, not just the labels.
Iterative problem-solving works. Testing must drive ongoing change, not mere compliance.
At Chaleit, we design testing programs that focus on what matters and actually make it harder for you to get breached.
Find out more about why our approach yields exponentially better results: Penetration Testing 3.0: Intelligence-Led Security Validation.
About the expert
Shawn Thompson is the Founder and Managing Director of Praexian Security Alliance. His work is dedicated to helping organisations move from complexity to clarity with a threat-informed, data-driven approach to cyber security.
