Hackers to Pentagon: You’re Doing Cyber Wrong

What happens when you bring together some of the nation’s leading hackers, the Pentagon’s chief of training and an Air Force Academy professor who teaches cyber skills to cadets? They all agree on one thing: The government’s approach to cyber security is coming up short.

They sat on the dais, an unusual assortment of experts at a conference for military simulation and training experts. No prepared speeches, just a wide open Q&A.

Their message in three bullets:

  • You can’t teach cyber defense without a thorough understanding and expertise in cyber offense
  • Cyber is all about breaking the rules. If you try to break cyber defense into a series of check-box requirements, you will fail
  • The Fifth Domain, as cyber is sometimes called in the military (joining air, land, sea and space) is not like the others. There is no high ground and the weapon you wield today may not even exist tomorrow

In the center was Frank DiGiovanni, director of Force Readiness and Training at the Pentagon, joined on his far left by Martin Carlisle, professor of computer science at the Air Force Academy. Sharing that stage were three of the best-known ethical hackers in the business: Jeff Moss, founder of Black Hat and DefCon, the two best-known annual hacker conferences; John Rigney, co-founder of Point3 Security, a Maryland cyber firm, who says he made his first hack at age 8; and Brian Markus, CEO of Aries Security, best known for his “Wall of Sheep” – an annual rite at the DefCon event, where he posts the names of all who have exposed themselves to security cyber hacks while attending the conference, which brings together some of the world’s top hacking talent.

What these five know about cyber security – or how to defeat it – can’t be cataloged. Indeed, part of their message is that cyber security, or cyber warfare, is so fluid, so rapidly evolving, that trying to define it or contain it is essentially impossible.

The government and industry are both in a quandary over the cyber challenge, partly because it’s unclear where their missions start and stop. America is fighting its cyber battles like the British fought in the American Revolution, he said. Back then, the British fought out in the open, following a well-drilled formula for combat. The Americans countered with guerilla warfare, fighting from the woods.

By limiting most of our defenders to defense-only approaches, the United States is effectively fighting while hand-cuffed. Cyber attackers, on the other hand, whether criminals or nation states, are playing without rules.

Said Markus: “We’re going up against a 300-pound fighter with one hand behind our back. We are going in with too many limitations.”

That’s the first thing cyber training needs to take into account: Cyber warriors have to be able to think like their attackers, and to do that they need to train like their attackers. Instead of focusing on rules and process, they need to focus on puzzle- and problem-solving.

Certifications are useful in understanding what people know, Carlisle said, but they are of limited use in fighting the active cyber attacker. Hackers buy cyber defense technology and then work on their own to defeat it. So one can’t be satisfied that having the best tools will be enough to protect your network.

The key to developing cyber talent isn’t to teach people to do well on certification tests, Carlisle and the others said, but rather to teach them to think and problem solve.

Said DiGiovanni: “If you think you can catalog every known thing that can happen to you, you’re wrong from the beginning…. To do this right, the training environ needs to be able to go beyond the square where you know exactly what you’re doing. The minute you do that, it’s exploitable. Someone will find a weakness in that training regimen and attack it.”

Similarly, Rigney questioned military efforts to standardize network design. “One attack profile means one target,” he said.

Cyber security is complex and fluid. Everything is changing, all the time, the panelists said. The military has the capability and the mission to develop the right tools. But to be successful, its policies and approach will have to change – and not just in how people are trained. One problem several panelists mentioned is that even when the military gets the training right, it sometimes mishandles the talent it produces.

This problem is not the military’s alone. Industry also makes mistakes. Markus described training programs to teach cyber skills that were highly successful, only to fail when it came to retaining talent. “When you train a bunch of people, and they get really excited, and amped up, and they have all this great knowledge of effects and warfare, and then you say, ‘Go watch the SOC [cyber operations center] logs,’ they say, ‘Fine, I quit. I want to go do something else.’ That’s why the industry is bleeding out people. They train them to be offensive warfare personnel, and then they have them go watch a gate.”

The military, everyone agreed, needs to be careful not to follow that model.

Carlisle, who emphasized he was speaking on his own behalf, and not for the Air Force Academy or the Air Force itself, said he rejects two common notions in military circles: First, that it’s ok to train for defense only out of concern about the risks involved with teaching people how to hack. “The military has certain fields, SEALs for example, who we accept that we train them to act with a certain degree of lethality. We should treat cyber the same way,” he said. Second, there are leaders who are satisfied to train cyber personnel only to lose them to industry after five or six years. Those leaders say what they really need are managers, not technical experts. “I reject that hypothesis,” he said.

The heart of cyber warfare, the panel agreed, is offensive operations. These are essential military skills they said, which need to be developed and nurtured – in order to ensure a sound cyber defense.


  1. Anonomyous

    “When you train a bunch of people, and they get really excited, and amped up, and they have all this great knowledge of effects and warfare, and then you say, ‘Go watch the SOC [cyber operations center] logs,’ they say, ‘Fine, I quit.
    It’s not surprising they quit, who wants to watch SOC logs? lol. You trained them to work on scenarios, not, to watch logs. Train someone else to watch logs, that sounds pretty procedural and straight forward to me. Relay the summary of that to these guys. Give these hackers a specific scenario to work on, and they won’t quit like that.

    • M

      It’s like the Guard and Active duty?
      Or like the infantry and the mobile reserve?

  2. Kenny

    Sad but true, spent time myself as a 25N.

    Wish they would have made the transition into cyber ware fare more available for those who show talent ,or at least dedication though the enlistment contract. However timing to personal is also an issue, restricting greatly who is trained and willing or able by keeping to the open slot system. ex. Only can take three people,now fight over it if you want in this year or often longer.

    The article at least read to me is how the branches are falling short in areas like self-or imposed restrictions, being unable to cross train, and adapt to the extreme level of variety and rate of evolving tech.
    Most notably however the comments of fighting with hands tied back, is this not a good thing? We might not want our cyber defense consolidated in the name of defense, to loose ourselves and loose precisely what we are trying to protect, or not recognize what we have become in the name of that security.

    That being said, with at least in my experience our millitary needs help. Perhaps adding more avenues to get to a level that they find useful or keeping slots rotating to give people fresh air if enlisted to keep people sharp. Many ideas out there to find ways of making this work. cohesion, adaptability and above all creating an environment with core values that spawned the hacker mentality.. Thirst for knowledge and how things work.

    Good luck to everyone out there, stay strong, hooah.


  3. Phillip Hallam-Baker

    The US ‘cyber defense’ program is almost entirely focused on attack. I have spoken to the people who set it up and they have no concept of cyber-defense. The US has not had to fight a defensive war in over a century.

    The fact that Snowden was able to roll the NSA over a year after Manning leaked all the state dept cables tells you where the DoD is on cyber defense.

    They are however world class at attack. If you look at the construction mechanism of FLAME and STUXNET it is obvious that there is an entire production facility set up for tailored cyber-weapons. If you read the Snowden papers you will find page after page of attack tools and almost nothing on defense.

    The hacker community has possibly developed techniques that were not previously known in the military. But remember that 90% of the NSA staff are civilians and all of them read the trade press, go to conferences, etc.

    The lone hacker has a completely different resource set and is forced to adopt an approach that focuses on ingenuity. The military mindset is to methodically search the entire space of the adversary’s capabilities looking for weaknesses.

    • Mike

      Mr. Hallam-Baker,

      With respect, you’re commenting as an outsider to the DoD. Yes, I’m aware of your background… how much has been within the DoD?

      The two posts above you (Anonymous and Kenny) are clearly commenting as people with experience inside the DoD. I also have experience inside the DoD.

      Most of what we do is classified as “Computer Network Defense” or “Defensive Cyberspace Operations” depending on which doctrinal term you leverage. See Joint Publication 3-12, available online. The NSA likely does some ops that most of us are not familiar with, but for the most part we defend.

      key fixes would to a world of good:

      1. Get Cyber leadership that understands what Cyber is capable of.

      2. Fix the rules for clearances such that we can recruit hackers with proven track records who may have been caught by the authorities at one point or another.

      3. For those ops that we do do, we need to leak details and take credit for them. Adversaries are not intimidated by silence.

      4. Break down the walls between the services and merge all DoD assets into a single command (think Joint Special Operations Command for Cyber).

      • Maria Horton

        Cyber is about evolving your response. Not just offense not just defense. Strategic protection may come from obfuscation or other means of camouflage. How we discuss and spin our strategies could help as well. Lots of other great comments from readers as well!

  4. Travis Pulley

    While reading the author’s credentials at the end of this piece, I’m disappointed that someone who should know better is citing the myth of the American Revolution being won with guerrilla tactics. Perhaps this speaks to the style of failures for which this article was written. If it’s so hard to phase out usage of the SHA-1 cipher in our military, what hope do we imagine? https://historymyths.wordpress.com/2013/05/18/myth-115-in-the-revolutionary-war-the-american-s-use-of-guerrilla-tactics-beat-the-british-who-fought-standing-in-straight-lines/

  5. Steven Nixon

    “The key to developing *cyber talent isn’t to teach people to do well on certification tests, Carlisle and the others said, but rather to teach them to think and problem solve.”

    *Officers, *NCOs, *Soldiers, *Paratroopers,*Rangers, *Business Managers, *Business Directors in my experience. I would have to defer to others’ experience, but suspect *Green Beret, *Operators, *Flag Officers, *C-Suite Leaders would work, as well.

  6. Frank Heidt

    In regard to the comments made by Phillip Hallam-Baker:

    I agree in large part with your assessment of the capabilities of the DOD, but only in so much as they align with the source of information you quote –most notably, some of the Snowden leaks.

    I would point out that as a Title 50 organization, the NSA — even though they are ostensibly part of the Department of Defense, are not, strictly speaking anything like a Title 10 entity in their mission — they’re not only code-makers, but also code-breakers. They both protect Title 10 institutions, and simultaneously keep critical vulnerability data from them.

    The entirety of the program that you point out as being focused with some exclusivity on attack would be more correctly attributed to Title 50 organizations like the CIA, DIA, and NSA, whereas the more mundane tasks of maintaining and defending DOD assets falls to commands like Army NETCOM, ARCYBER, DISA &c.

    The NSA does not run defensive operations at the Camp, Base and Post level, they set some standards, provide some valuable tools, and on rare occasion direct testing capabilities. Day-to-day defense is delegated to the various branches of the armed forces, that’s one of the primary reasons the Snowden archive is heavy on IA and light on IO.

    • Frank Heidt

      Clearly I transposed IA and IO on my last line, please accept my correction: Heavy on IO (Information Operations) and light on IA (Information Assurance)

  7. Mike Matney

    Unfortunately there are very few who see DoD’s “big picture” and know what the military’s overall cybersecurity strategy. Some of the supporting ideas of this article are more a part of the military’s hard formed processes, and will be very hard to change. As far as the three central ideas, I agree with the first bullet, somewhat agree with the second and violently disagree with the third bullet. The first bullet is well understood by DoD and this understanding is in our framework. The second bullet, check-block processes do work, maybe not in a current tactical engagement, but our strategy and operations are very much informed by these tactical fights. Some examples of the inputs to these “processes” include intelligence, operations, planning, R&D, tool life cycle etc: Drilling further down the intelligence category. We define a thorough understanding of our adversary including the type, size, (some cases down to the team) that conducted the infiltration/exploitation and understand the tactics, techniques and procedures they use. We use intelligence to better prepare for future operations by tuning our defensive systems across avenues of approach. These check-box requirements make the ability to plan and conduct defensive operations better. This collection is used in order to 1. Take direct action / disruption / deny / degrade adversary offensive operations in flight, 2 make operational plans for future attacks, 3. conduct network denial operations or 4. conduct denial operations for example. All of which are processes. Finally the last bullet about cyber being different than the other domains, try walking into a Combatant Commander’s update with that approach. Let me be a fly on the wall when you brief how “special” cyber is and how it is not the same as the other domains. You will not win any friends on the staff and are missing the point of joint, interagency, intelligence community, multinational and unified operations.

    • Mike Matney

      Apologies, should have used destructive under point #4.

  8. SPEG

    I think another issue that is overlooked is attitude. People just don’t take cyber security seriously. I have received countless E-mails at home purporting to come from legitimate sources that tell me my bank account will be locked, or some other trouble will befall me if I don’t click on their link . When I hover over the URL, it’s from weird, goofball off-the wall address that bears no resemblance to the legit ones. I call them “Vampires”. They can’t come in unless you “invite” them in. How many attacks in government and industry started by somebody, regardless of rank, position, or status clicking on a link to a malicious site?
    Most employers and all government agencies have mandatory training for their employees on how to avoid such missteps, yet they continue.
    Everyone needs to consider themselves a cyber warrior of sorts. Whether on the font lines in some national agency’s operations center or simply in an office and being watchful of their E-Mails. The attitude, up and down the chain of command and responsibility is just not there.
    I pray that a cyber “Pearl Harbor” doesn’t happen before leadership wakes up.


Submit a Comment

Your email address will not be published. Required fields are marked *

Related Articles

GDIT Recruitment 600×300
GM 250×250
GDIT Recruitment 250×250
Vago 250×250
GDIT HCSD SCM 2 250×250 Plane Takeoff
(Visited 19,376 times, 1 visits today)