Paradise: Navigating by Touch When You Can't See or Hear
We spent the weekend blindfolding our teammate, putting noise-cancelling headphones on him, and sending him across a parking garage at midnight with two Joy-Cons strapped to his wrists. He found the car. Mostly.
It's midnight at ShellHacks and we're in a parking garage. My teammate is standing there blindfolded with noise-cancelling headphones on and a Nintendo Joy-Con strapped to each wrist. I'm about forty feet away, next to a car. Nobody is allowed to say anything, because the entire point is that he can't hear us.
His left wrist buzzes. He turns left. Both wrists buzz. He starts walking. Thirty seconds later his hand is on the car door.
That's Paradise. It took 3rd overall at ShellHacks 2026 out of 300+ projects and 1,400+ hackers, which lowkey still hasn't sunk in. But the demo is the easy part to write about. I want to talk about the stuff that almost didn't work.
The gap nobody builds for
About 50,000 Americans are fully DeafBlind. Around 2.4 million have serious vision and hearing loss together.
Here's the part that got me. Almost every navigation app built for blind people talks to you. Almost every app built for deaf people shows you something. If both of those are out, you don't get a worse version of the app. You get nothing. The whole category just stops.
So we set one rule: no sound, no screen. Everything arrives through skin.
The entire language is five signals
- Left wrist buzzes → turn left
- Right wrist buzzes → turn right
- Both wrists buzz → walk forward
- Double buzz on both → you're there
- Nothing → don't walk
That's it. You can learn it in about thirty seconds, which we tested on basically everyone who walked past our table.
The non-obvious decision is that we built more patterns and then deliberately turned them off. There's code in there for an obstacle stop (three pulses), a searching pattern, a found-it pattern, a choice echo that pulses N times. All of it works. All of it is switched off.
Because the second you have eight signals instead of five, the person has to stop and think. And a vibration you have to decode is worse than no vibration at all when you're standing in a parking lot. A small vocabulary is faster to learn and much harder to confuse. That was probably our best call of the weekend, and it was a call to delete things.
Why Nintendo Joy-Cons
People asked this constantly and the answer is boring: two independent rumble motors, Bluetooth, physical buttons, and you can buy them today at Target. A real product would be a custom wristband. We had a weekend.
The annoying part is that driving them from a browser needs WebHID, which is Chrome-only. iPhone Safari can't do it. So the system ended up split across three devices: an iPhone strapped to your chest doing the seeing, a MacBook in a backpack doing the heavy thinking and running the Joy-Cons, and a third phone acting as a beacon that pretends to be the car. There's no public Waymo API, so somebody just has to stand there holding a phone.
It is not elegant. It worked.
The bug that steered us into a wall
This one cost us hours and it's my favorite thing I learned all weekend.
GPS tells you where north is. The iPhone compass also tells you where north is. They are not the same north.
GPS uses true north, the actual top of the planet. A compass points at magnetic north, which is somewhere else entirely, and how far off it is depends on where you're standing. In Miami it's about 6.8 degrees.
Six degrees sounds like nothing. It isn't. Over 40 meters, 6.8 degrees puts you about 5 meters off target, which is a lane of traffic. And because the error is consistent, it doesn't look like a bug. It looks like the system confidently walking you slightly wrong, every single time, forever. Those are the worst ones to find, because nothing ever crashes.
The fix is one number in one file. Finding it is the entire job.
Using the floor as a ruler
Next problem. We're running YOLO, which looks at a camera frame and draws boxes around things it recognizes. It knows 80 everyday objects: person, chair, bottle, car.
Walls are not on that list. Neither are poles, or the concrete pillar in the middle of the garage. YOLO will happily walk you straight into a wall, because as far as it's concerned there is nothing there.
So we added a depth model, Depth Anything V2, which guesses how far away every single pixel is. That solves it, except for one thing: a single camera has no sense of scale. It can tell you pixel A is farther than pixel B. It cannot tell you whether you're looking at a real hallway or a very good photo of a hallway. And iPhones don't let web pages touch the LiDAR sensor, so we couldn't cheat our way out.
What we did instead was find the floor and use the floor as the ruler. The system fits a flat plane to the ground, and once you know where the floor is and roughly how high the camera sits above it, "farther than" turns into actual meters. Anything between 12cm and 2.1m above the floor, inside a narrow corridor directly ahead, between 0.3 and 4 meters out, counts as an obstacle. Stop at 0.8m. Walk around it between 0.8 and 2.5m.
The floor trick works well. It also means if you type your chest height in wrong, every distance in the entire system is wrong, which is a genuinely fun thing to debug at 3am.
We had to train our own Waymo detector
Finding a car is easy. Finding the car is not.
First approach was open-vocabulary detection, where you literally type "Waymo sensor dome" in plain English and the model finds it. Grounding DINO does this and it's very cool.
It also scored car side mirrors higher than actual sensor domes. Repeatedly. A mirror is a shiny rounded thing sticking off a car, so you can see how it got there. But "you can see how it got there" doesn't help a blindfolded person walking toward the wrong vehicle.
So we trained our own. We took CLIP, which turns an image into a list of 512 numbers describing what's in it, and put a tiny classifier on top that learns which combinations of those numbers mean Waymo. Training data was 125 photos of Waymos (the Jaguar I-Pace, the Pacifica, the Zeekr) and 654 photos of other cars, deliberately including look-alikes.
It lands at 89% recall and 96% precision, about 4.7 false positives across 654 non-Waymos, and it runs in 11.6ms, which is faster than everything else in the stack by a mile.
The detail I'm actually proud of is the validation. We grouped the cross-validation folds by source photo, so two crops from the same original image can never land on both sides of the split. Without that, the model gets to see a corner of a test image during training, and your numbers come out beautiful and completely fake. Grouping cost us a couple of points on paper and made the number mean something. Tbh that's the part I'd want someone to ask me about.
Stuff that is still broken
I'd rather write this section than skip it.
The door handle finder never fires. We built it. Grounding DINO loads, it's sitting right there in the repo, and it has never once run in a real session. The car search hands off to close-range detection at 25 meters, but the handle detector doesn't open its window until 4 meters. The handoff happens before the thing that needs it is allowed to start. Pure sequencing bug, and we ran out of hours.
Hand-reaching is turned off. There's a flag in the code, REACH = false. MediaPipe hand tracking is wired up and would guide your actual arm onto the handle using higher and lower buzzes for up and down. Disabled. Arrival just ends guidance and you're on your own for the last 30cm, which is exactly the part that's hardest.
Silence means too many things. This is the design flaw that bothers me most. Right now "nothing" means: waiting, stopped, obstacle ahead, signal lost, and finished. Five different states, one sensation. If you're DeafBlind, the device going quiet should never be ambiguous, and right now it completely is.
The depth model can't see the dangerous stuff. Low curbs around 15cm, steps going down, holes, glass doors, overhangs. All invisible to it. Which is to say, the things most likely to actually hurt someone are the things it's worst at.
We have not tested this with a single DeafBlind user. Everything so far is simulations, photo tests, benchmarks, and us blindfolding each other in a parking garage. That makes it a prototype, not a product, and I don't want to describe it as anything more than that.
The conversation that changed it
One of the judges, Anudeep Goud, spent a while pushing us on who we were actually building for. Not in a gotcha way. He just kept asking questions we didn't have clean answers to. Who's holding the beacon phone in real life. What happens when the device goes quiet and you don't know why. Whether a sighted person's idea of "arrived" is the same as arriving when you can't see anything.
We changed how we talked about the project after that, and honestly how we thought about it. Best fifteen minutes of the weekend.
The billboard
On the drive down to ShellHacks we passed a van that said Paradise Garage Doors. About a mile later, a billboard: "Paradise is only 10 minutes away."
We hadn't named it yet. We took it as a sign. Turned out to be right.
Built with Akhilesh Reddy Mallu, Devam Dholakia and Pranavsai Gandikota. Three hackathons, three wins this year, and I genuinely could not have done any of them on my own.
There's a demo video, and the full writeup is on Devpost.