3. We Thought we
Understood the
Problem
We Achieved Product-
Mission Fit
No Solutions
Worked
The Right
People are
Interested
We Believe this
could be a
Viable
Business
PIVOT
PIVOT
Found Existing Solution
for Waypoint Navigation
4. MISSION BUDGET/COST (ordered by magnitude)
1. Materials for prototyping and R&D: sensors, structural components, tools
(number of materials can be loaned at zero cost from key resources)
2. Testing
3. Computing resources
4. Data access (NASA gravity, US government, Stanford GIS, Descartes/Planet GIS)
DEPLOYMENT
- Test flight at Moffett Federal
Airfield, Mountain View
- Mass production by
integrating to General
Atomic, Lockheed Martin,
Boeing, and Northrop
Grumman assembly lines
BUY-IN & SUPPORT
DoD Joint Artificial Intelligence
Center: approve the design and
prototype
Manufacturers: adopt design,
mass produce our product, and
integrate it into their existing
systems
KEY RESOURCES
- Access to beneficiaries
- Stanford faculty
- Experts on drone warfare
- Joint Artificial Intelligence
Center
- Access to self-driving, drone,
and IoT tech in Silicon Valley
KEY ACTIVITIES
- Framing the problem
- Conducting research
and speaking with
experts (military,
academic, and industry)
- Designing solution
- Prototyping + Testing
KEY PARTNERS
- US Air Force
- H4D team
- Joint Artificial
Intelligence Center
- Stanford Research
Labs (AA, geophysics
departments) and
Stanford Community
- Drone manufacturers:
General Atomic,
Lockheed Martin,
Northrop Grumman,
Boeing
MISSION ACHIEVEMENT/IMPACT FACTORS
- Design approved by DoD or defense manufacturers
- Deployment on at least one military-grade UAV and when tested, features with
favourable processing time compared to current systems, reduced location
uncertainty, improved F1 score on detections, and full functionality in GPS-
denied environments
We Started by Understanding Our Beneficiaries
VALUE PROPOSITIONS
1. Reduced Risk of
Drone Loss
1. Higher quality data
and better decisions
1. Lower risk of
detection
BENEFICIARIES
● USAF UAV pilots
● USAF UAV
commanders
● UAV
manufacturers for
DoD
We Started by Understanding Our Beneficiaries
MISSION BUDGET/COST (ordered by magnitude)
1. Materials for prototyping and R&D: sensors, structural components, tools
(number of materials can be loaned at zero cost from key resources)
2. Testing
3. Computing resources
4. Data access (NASA gravity, US government, Stanford GIS, Descartes/Planet GIS)
DEPLOYMENT
- Test flight at Moffett Federal
Airfield, Mountain View
- Mass production by
integrating to General
Atomic, Lockheed Martin,
Boeing, and Northrop
Grumman assembly lines
BUY-IN & SUPPORT
DoD Joint Artificial Intelligence
Center: approve the design and
prototype
Manufacturers: adopt design,
mass produce our product, and
integrate it into their existing
systems
BENEFICIARIES
● USAF UAV pilots
● USAF UAV
commanders
● UAV
manufacturers for
DoD
KEY RESOURCES
- Access to beneficiaries
- Stanford faculty
- Experts on drone warfare
- Joint Artificial Intelligence
Center
- Access to self-driving, drone,
and IoT tech in Silicon Valley
KEY ACTIVITIES
- Framing the problem
- Conducting research
and speaking with
experts (military,
academic, and industry)
- Designing solution
- Prototyping + Testing
KEY PARTNERS
- US Air Force
- H4D team
- Joint Artificial
Intelligence Center
- Stanford Research
Labs (AA, geophysics
departments) and
Stanford Community
- Drone manufacturers:
General Atomic,
Lockheed Martin,
Northrop Grumman,
Boeing
MISSION ACHIEVEMENT/IMPACT FACTORS
- Design approved by DoD or defense manufacturers
- Deployment on at least one military-grade UAV and when tested, features with
favourable processing time compared to current systems, reduced location
uncertainty, improved F1 score on detections, and full functionality in GPS-
denied environments
VALUE PROPOSITIONS
1. Reduced Risk of
Drone Loss
1. Higher quality data
and better decisions
1. Lower risk of
detection
5.
6.
7.
8. No Solutions
Worked
We Thought we
Understood the
Problem
PIVOT
PIVOT: We shifted focus to oversea, waypoint navigation only.
9.
10.
11. No Solutions
Worked
We Thought we
Understood the
Problem
PIVOT
PIVOT
Found Existing Solution
for Waypoint Navigation
PIVOT: Skymark’s 100m accuracy leaves the overland target-area
navigation problem unsolved, so we shifted our focus there.
15. In each frame, satellite imagery and
SAR topographic matching outputs
lat-long coordinates
Frames are overlaid, and visual
odometry outputs velocity and
acceleration
16.
17. We Thought we
Understood the
Problem
No Solutions
Worked
We Achieved Product-
Mission Fit
PIVOT
PIVOT
Found Existing Solution for
Waypoint Navigation
18. MISSION BUDGET/COST (ordered by magnitude)
1. Materials for prototyping and R&D: sensors, structural components, tools
(number of materials can be loaned at zero cost from key resources)
2. Testing
3. Computing resources
4. Data access (NASA gravity, US government, Stanford GIS, Descartes/Planet GIS)
DEPLOYMENT
- Test flight at Moffett Federal
Airfield, Mountain View
- Mass production by
integrating to General
Atomic, Lockheed Martin,
Boeing, and Northrop
Grumman assembly lines
BUY-IN & SUPPORT
DoD Joint Artificial Intelligence
Center: approve the design and
prototype
Manufacturers: adopt design,
mass produce our product, and
integrate it into their existing
systems
KEY RESOURCES
- Access to beneficiaries
- Stanford faculty
- Experts on drone warfare
- Joint Artificial Intelligence
Center
- Access to self-driving, drone,
and IoT tech in Silicon Valley
KEY ACTIVITIES
- Framing the problem
- Conducting research
and speaking with
experts (military,
academic, and industry)
- Designing solution
- Prototyping + Testing
KEY PARTNERS
- US Air Force
- H4D team
- Joint Artificial
Intelligence Center
- Stanford Research
Labs (AA, geophysics
departments) and
Stanford Community
- Drone manufacturers:
General Atomic,
Lockheed Martin,
Northrop Grumman,
Boeing
MISSION ACHIEVEMENT/IMPACT FACTORS
- Design approved by DoD or defense manufacturers
- Deployment on at least one military-grade UAV and when tested, features with
favourable processing time compared to current systems, reduced location
uncertainty, improved F1 score on detections, and full functionality in GPS-
denied environments
VALUE PROPOSITIONS
1. Reduced Risk of
Drone Loss
2. Ability to reach
target
3. Lower risk of
detection
4. Improved
strategic reach
BENEFICIARIES
1. USAF UAV pilots &
Fighter Pilots
1. Mission Commanders
MISSION BUDGET/COST (ordered by magnitude)
1. Materials for prototyping and R&D: sensors, structural components, tools
(number of materials can be loaned at zero cost from key resources)
2. Testing
3. Computing resources
4. Data access (NASA gravity, US government, Stanford GIS, Descartes/Planet GIS)
BUY-IN & SUPPORT
DoD Joint Artificial Intelligence
Center: approve the design and
prototype
Manufacturers: adopt design,
mass produce our product, and
integrate it into their existing
systems
KEY RESOURCES
- Access to beneficiaries
- Stanford faculty
- Experts on drone warfare
- Joint Artificial Intelligence
Center
- Access to self-driving, drone,
and IoT tech in Silicon Valley
KEY ACTIVITIES
- Framing the problem
- Conducting research
and speaking with
experts (military,
academic, and industry)
- Designing solution
- Prototyping + Testing
KEY PARTNERS
● H4D team
● Joint Artificial
Intelligence
Center
● 432nd command
● Planet Labs
● ISR Panel
● Air Superiority
Panel
● Draper Labs
● General Atomics
With Product-Mission Fit, We Shifted Our Focus to Deployment
DEPLOYMENT
1. Develop software-only
solution with NSF funding
2. Mature tech with MQ-9 PO
funding and support
3. Purchase, Integration, and
Testing by MQ-9 PO and
General Atomics
DEPLOYMENT
1. Develop software-only
solution
2. Mature tech with MQ-9 PO
3. Purchase, Integration, and
Testing by General Atomics
19.
20. Going forward...
- Abhay, Stefano, and Freddy will continue
- Incorporate
- Software development
- STTR Funding
- Exploring dual-use + VC Funding down the line
- Testing
- Start deployment process in a year
Contact us at: team_helmsman@lists.stanford.edu
21. Going forward...
- Abhay, Stefano, and Freddy will continue
- Incorporate
- Software development
- STTR Funding
- Exploring dual-use + VC Funding down the line
- Testing
- Start deployment process in a year
Contact us at: team_helmsman@lists.stanford.edu
Editor's Notes
Good evening. We are Team Helmsman, a group of 3 freshmen and a masters student sponsored by the Joint Artificial Intelligence Center, better known as the JAIC. Starting off, we knew nothing about the problem. All we had was a problem statement saying to eliminate electromagnetic emissions from military drones and fly in GPS-denied environments. However, after 101 interviews, our understanding of the problem changed, and we now know large military drones need to fly with high-precision navigation near their target in GPS-denied areas to complete their mission.
Through the span of these 10 weeks, our 101 interviews included 28 active UAV and fighter pilots, 15 navigation experts from leading research institutions, 30 military contractors and UAV manufacturers, and 14 program managers, acquisitions, and requirements officers.
Our journey was a roller-coaster ride with multiple pivots as our understanding of the problem evolved, our focus shifted, and our ideas for solutions hit dead ends. In this presentation, we will take you through this journey.
The first week, we had a lot of hypotheses, and we started by first understanding our beneficiaries: their routines, their jobs, their pains, and their gains.
Early on during the beneficiary discovery process, our problem sponsor Major Daniel Tadross in the JAIC precisely summarised that “the problem is that UAVs have no way to navigate in GPS-denied environments”
Digging deeper, we learned that Challenges in UAV Navigation Comprise Two Segments: Overland and Oversea. In the representative mission, after the UAV takes off, first it must travel long distances over water navigating waypoint-to-waypoint before reaching the target area. At this stage, it requires approximately 100 m precision in its coordinates; as long as it's going in the right direction it will get there. Once the UAV reaches target area, it enters the next navigation segment: terminal phase navigation over land. At this point, precision requirements go up a bar, as position must be known to no less than 1m of error
We were optimistic and tried to solve the problem as a whole, but as we explored solutions, we only hit dead ends. Existing magnetic field maps are not accurate enough for navigation. Dedicated radio signals like LORAN are jammable and were decommissioned. Signals of opportunity are geographically dependent and thus unreliable, and gravimetric sensors are not accurate enough to navigate by gravity.
Our morale sunk as no solution worked, so we realized we had to pivot and pick one part of the problem to solve. We focused on oversea, waypoint navigation.
Per a discussion with senior engineers at BAE Systems, as target area navigation first requires the aircraft get there, oversea waypoint navigation was the problem to focus on. Over water, we discovered that celestial navigation met pilots’ requirements.
However, as we dug deeper into celestial navigation, we found we were right in going that path, but we were too late. We were surprised and disappointed to find that Draper Labs’ Skymark had already virtually solved the overwater problem, checking off requirements. As an established military contractor with piles of funding and a 4 year head start, they indicated they would soon deploy.
We were bummed, but still persisted. Another opportunity opened. With waypoint navigation addressed, we pivoted to the problem of overland target area navigation, which remained open.
With one path open, our new challenge was to navigate with 1-meter location precision within the target area over land. With new environments and new requirements, a number of new solution opportunities appeared.
Focusing overland, we looked to the past to find solutions for the future. In the past, terrain contour matching, TERCOM, and digital scene-mapping area correlator, DSMAC, worked by imaging the ground below, through radar and conventional cameras respectively, and later comparing these images to an onboard georeferenced database.
However, TERCOM and DSMAC restricted flight paths due to limited processing power, low accuracy, and limited image databases. Hence, GPS, which was cheaper, more accurate, and more reliable, replaced them. With time, these technologies were forgotten.
Nowadays with faster processors, better cameras, higher resolution maps and modern algorithms, a modern version of TERCOM and DSMAC can become an unjammable replacement of GPS.
_________
Faster processors, better cameras, higher-resolution maps, modern algorithms
GPS Greater Precision, More Reliable, Lower Cost Solution
No one was blocking GPS
Everything old is new again
Modern TERCOM measures change in altitude.
Improvements in computing and memory, combined with the availability of global digital elevation maps, has reduced this problem, as TERCOM data is no longer limited to small patches, and the availability of side-looking radar allows much larger areas of landscape contour data to be acquired for comparison with the stored contour data.
During the gulf war tomahawk missile were using DSMAC in the place of GPS because higher precision at the time.
Our solution, Helm, does exactly that. It brings TERCOM and DSMAC into the 21st century with modern technology.
As you may remember from the video, once a UAV enters a GPS-denied environment it starts taking pictures off the ground.
Combining satellite imagery and topographic mapping, Helm matches the photographed image to the onboard satellite image database providing accurate lat long coordinates.
For greater robustness, this tech is combined with visual odometry, a paradigm to track movement relative to the ground, which reinforces the onboard inertial navigation system.
Our MVP overlays pictures of the ground, applying these three algorithms to accurately position itself within target area, guaranteeing 1m precision positioning throughout its path.
With an MVP in mind, we reached out to our beneficiaries once more. Their feedback was very positive.
One of them even mentioned:
“You are making missions possible. With reliable precision positioning, pilots can locate targets and complete mission without GPS.”
With this feedback suggesting product-mission fit, team morale soared!
And so, once we had a desirable solution, we shifted our focus to deployment.
We found key partners in industry and interested buyers in the Air Force and Navy who can help take our MVP to deployment. General Atomics has already offered to share engineering expertise and unclassified MQ-9 platform specs. Planet has offered cheap, high-resolution satellite imagery in bulk.
With this support and deployment promise, Stefano, Freddy and myself want to incorporate and deploy our MVP. In the near future, we will build our software solution with STTR funding and integrate into existing hardware. Then, after exploring dual-use applications, with potential VC funding, we intend to rigorously flight test and finally deploy our solution to address a key part of GPS-denied navigation.
With this support and deployment promise, Stefano, Freddy and myself want to incorporate and deploy our MVP. In the near future, we will build our software solution with STTR funding and integrate into existing hardware. Then, after exploring dual-use applications, with potential VC funding, we intend to rigorously flight test and finally deploy our solution to address a key part of GPS-denied navigation.
On a concluding note, we would like to thank all those who shared their time and expertise with us, with a special thanks to our business mentor Todd Basche, our military liaison Lt Col Angie Waters, key supporter Marco Romani, and of course our sponsor Major Daniel Tadross,.
The ideal thing to do in our video is to really drill into the problem and the solution so that people have already seen the problem in video
“We’re Team Helmsman and we were asked by the JAIC to find a way to navigate UAVs in GPS-denied areas”. Then go right to the whole thing. “We looked at all kinds of solutions: overland and overseas but our successful MVP came from looking to the past.” Describe TERCOM and DSMAC for a few seconds and add a visual of an old plane using that. “But today we have modern XYZ, and our solution will be to combine XYZ” and demonstrate how the MVP works
We can include the video montage after our introduction. “Recent history has shown that drones are lost…”
A two minute narration of our solution and where the technologies came from and stitching it all together to create a map to get to target would be a brilliant use of the two minutes. The fact that Steve Blank wasn’t getting our MVP is a problem, and we need to fix that.
If we can find videos of Tomahawks and MQ-9s and pilots, they’re cool visuals we could show as we describe the problem and solutions. Don’t waste time on Draper and what we didn’t do. Being able to have that animation and explain what’s happening is a money shot.
He didn’t understand the VIO video. It’s a technical part of the solution that’s not as interesting as the narrative “we looked at the past and developed this technology that can do this.” It’s a super cool solution, but no one in the teaching team got it.