20 Minutes to Impress: Inside DIA’s Innovation Hub
The make-shift command center is large and bright, tiered so that those in the back can see more clearly. Overhead projectors light up the long wall in front, and additional flat screen monitors flank the projected images on the wall. The room is large enough to accommodate as many as 50 people, but on this day only about a dozen are present, scattered about the room, looking for innovation, hoping to the two presenters waiting nervously to get started will spark some new idea.
This is the Defense Intelligence Agency’s (DIA) Innovation Hub, the center of an effort intended to acceleration the pace of change and innovation in the agency. The two presenters will get just 20 minutes to show what their product can do and to earn a chance to return for a longer demo and test and – they hope – a development contract. The presentation will take less time than it takes to get through security.
DIA first launched its Innovation Office three years ago and has refined its process over time. Needs are posted on the agency’s NeedipeDIA web page, and interested parties are invited to submit brief white papers describing their potential solutions. The most promising ones earn an invitation to present at an industry day like this one.
DIA held a one-day event with seven demonstrations in December, all focused on a single need – a “User-Defined Intelligence Picture” intended to enable intelligence analysts to individually configure monitors to view multiple intelligence sources simultaneously; two of the seven have moved on to contract talks. Then, in April, the agency held its first two-day innovation event, receiving 32 white papers, of which 20 were deemed worth a further look. Three were dropped after pre-briefs, leaving 17 vendors for the event. Their technologies addressed advanced analytic support, tech identification, intelligence on weapons of mass destruction, threat tracking, discrete communications, electronic signatures, bulk translation, surveillance and counter surveillance, cyber behavior analytics and human persona understanding.
Agency partners dialed in from out of town to participate via video teleconferencing, and DIA says future demos may be opened to even more partners.
For the demonstrators, it’s nerve racking. The brief time limit means they have no time to waste and the pressure is on from the moment they start. The rules are strict: Visitors may not exchange business cards or talk to DIA participants except as part of their demonstration; PowerPoint presentations are forbidden; questions from the floor are expected and encouraged.
For this one instance, a reporter is allowed in to watch one of the demonstrations, a learning analytics platform that uses neural networks to break problems down into pieces, then compute answers with mind-boggling speed. But within minutes of its start, it is clear the demo is in trouble. After a brief introduction, the demonstration times out.
“This might take a few seconds,” one of the demonstrators says, laughing nervously. The room goes silent, waiting for the hang-up to resolve itself. It doesn’t. Seconds become minutes. “It worked in the parking lot,” he says. More nervous laughter, more silence. It’s painful to watch. The DIA people seem understanding and the demonstrators try to go with the flow, but it’s hard. Seeing is believing and there’s nothing to see. They fall back on describing how it works and then the questions start.
“It’s not parallel computing, is it?” asks a DIA staffer in back of the room.
“Yes it is,” he’s told. The system is designed to break down problems into discrete pieces to accelerate the number crunching.
“I can appreciate the distributed nature,” says a second DIA observer, a woman in the front. “But for the use case of technical identification, how will this platform help me identify unknown unknowns?”
The platform supports machine learning by running algorithms better and faster than conventional computing technology, but it’s not magic. It’s only as good as the algorithm it supports. The discussion shifts to machine learning and what it takes to train the system, and that prompts another question: “How would it prevent confirmation bias?”
“That depends on the questions you’re feeding into the algorithm.” Interest in using machine learning to tackle the vexing question of unknown unknowns is growing, he says. Algorithms can be chained together to take on increasingly complex concepts. But there is potential to introduce “machine bias” – that is, the sense that the computer must be right, even though the answers are based on human inputs.
When the 20 minutes are up, the session ends. The presenters say thank you and depart for the coffee room, disappointed but confident that it was worth the effort. “These things are forcing functions. Without a reason to present, we might keep researching and developing forever. We learned something here.”
DIA officials say they’ll try to arrange another demonstration, perhaps at the vendor’s offices instead. But it’s hard to escape the disappointment over the demo.
Innovation is hard. Even showing off a technology can be hard. In a classified environment, insulated from the outer world, it’s even harder. There can be a disconnect between the real-world hands-on problems of the analysts and commercial technology providers that may not have a full grasp of what’s needed inside the building. At the same time, the experts inside agencies do not necessarily know what’s technologically possible on the outside.
The Innovation Office was created to help bridge that gap, and three years into its development, Robert Dixon, Jr., special advisor for programs and transition in DIA’s Innovation Office, says the agency has matured and improved its formula. “You have to transform the culture and build partnerships across the enterprise to make this work,” Dixon says. “And you have to pay attention to what works and what doesn’t and keep evolving, too.”
Connecting with commercial industry is a particular challenge for experts in the sheltered intelligence world. The “NeedipeDIA” requirements page aims to help, providing an open invitation to vendors to share their ideas, Dixon says. Once shared, the Innovation Office can help vet those concepts with internal customers and facilitate interaction with industry days and technology demonstrations. The most promising ideas can be acquired for testing on an isolated replica network – a lower bar than going through the testing and review necessary to take a new piece of gear and hook it up to the classified intelligence network.
In an organization where procurement is often measured in years, this process is designed to go from concept to pilot in just months, Dixon says.
DIA is also trying to share with its partners, working closely with U.S. Central Command, U.S. Pacific Command and the Pentagon’s Defense Innovation Unite Experimental (DIUX), and in the future with other intelligence agencies and possibly select foreign partners.
“The beauty is it’s unclassified,” Dixon says. “We can share with our partners and let them pilot solutions with us.”
Showing the Money
One big change is financial. DIA’s early innovation efforts ran into funding difficulties because mission partners were unable to come up with the needed research and development funding, Dixon says. The Innovation Office identified worthy ideas, but then mission owners couldn’t deliver funding to support them. General Dynamics Information Technology (GIDT) was among those early winners, presenting a concept tying facial recognition to social media for intelligence operators. DIA operators liked it, but no funding ever emerged.
The problem: DIA needed a mission owner to step up with research and development funding to advance such projects, but the mission owners didn’t have the funds or flexibility to make that happen. The Innovation Office didn’t either.
Now the Innovation Office is looking at a different funding framework, one that can leverage operations and maintenance (O&M) funds, as well as R&D. O&M funds are easier to come by. “That will give us more flexibility,” he says.
Each demo and each pilot is an opportunity to learn as much as possible as quickly as possible, Dixon says. “We want to learn in one pilot what we can, then leverage that knowledge with a second pilot,” Dixon says. The idea is to prove the concept fast and with a small investment; if it fails, that’s ok, because the investment in time and cash was small. But if it shows promise, then the process can continue. When a mission partner sees promise and champions the solution, the concept can evolve into a program of record.
The Innovation Office, meanwhile, will measure its success one effort at a time, measuring its effectiveness each step of the way. “How relevant was the pilot to the need? What came out of this that we maybe didn’t anticipate from the beginning?” Dixon says, posing the questions that emerge after each pilot. “What type of efficiencies could we achieve, in terms of cost savings or time?”
The office has the full backing of Marine Lt. Gen. Vincent Stewart, DIA’s director, Dixon says. “He’s very committed. He told us he wanted the IHub set up within 30 days and he’s given us the resources to build a better one. This is a priority.” By June, he said, a new innovation hub will be in place.
Dixon says NeedipeDIA has helped DIA engage over 120 private sector innovators, with more than 20 percent of those earning a chance to present their ideas in person. The exchanges can yield follow-up engagements and broader discussions, which in turn can develop into pilots or prototype programs. In each case, Dixon says, the agency tries to answer fundamental questions: “Can we scale this across the enterprise? Can we sustain it?”
Although most participants have been small companies, large prime contractors have also participated. Indeed, Dixon acknowledges that traditional contractors shared history with the intelligence and defense communities give them an edge in some cases, because they have an innate understanding of existing capabilities and potential needs. Small startups may have valuable capabilities, but may have neither the understanding of how it would fit into intelligence requirements nor the patience for the pace of government contracting – even when it’s accelerated, as the Innovation Office intends.
Says Dixon: “It’s not the size of the business that’s important. It’s the idea. We’re casting the net wide, to anyone who has the solutions that can help solve our problems. It could be someone working out of a garage someplace or it could be a large industry partner.”