Return to the AccessAbility Podcast Page
Post-Production FILE
AAW 013 Rizzo.plain_doc.docx
March 10, 2022
Transcription is provided in order to facilitate communication accessibility and may not be a totally verbatim record of the proceedings.
* * * * *
[MUSIC PLAYING]
Welcome to AccessAbility Works, a podcast about the possibilities of accessibility for people with disabilities. I’m Jonathan Hermus.
And I’m Albert Rizzi, and this is the AccessAbility Works podcast. I’m looking forward to today’s podcast, Jonathan. We are going to be speaking with friend and colleague JR. Rizzo, Dr. Rizzo, JohnRoss Rizzo, the Rizz master. And he is a professor, and researcher scientist inventor at NYU.
Your ancestors come from the same place?
I don’t know, but he spells his name funny.
One letter off.
But the one thing about JR. and I is, we always used to be a radio and technology store TVs, turntables, back when we had those. And their logo was, nobody beats the Wiz. And in this instance, I can honestly say that nobody beats the Rizz, because JR. and I both work really diligently and hard to level the playing field for people who are blind, visually impaired, and/or print disabled.
And he’s also the director of innovation and technology for the Department of Physical Medicine and Rehabilitation at Rusk Institute of Rehabilitation Medicine. When Dr. Rizzo was a young boy, he was diagnosed with choroideremia.
Try and say that 10 times.
I’m not going to try and I say it perfectly, though, I think. Choroideremia.
I’m not sure what that is or how it does–
The way JR. talks about his condition, is it is it a visual impairment that limits his peripheral vision, I believe, or his central vision. He’ll get into it more, but it affects the way the eyes process the information. So he does have some usable vision, but not enough to get by.
So he’s in a very unique position on having this disease or condition.
Condition, depending on how you look at it.
And he’s able to research it for himself.
Well, that’s the thing going back to nothing about us without us. We are stakeholders in the outcomes and JR has a very vested interest in all that he is doing at NYU to harness the power of technology to give us the information that we need to process devoid of sight. So it’s really an interesting wearable that he’s going to be telling us about, and let him talk about that when we get there.
But before we get to the interview, just a bit of stuff that we did over the past few weeks. Jonathan and I took a quick getaway to Costa Rica. Visited friends of ours down there. And had a great time hitting sites and was one of the first blind people to zip line at the location we went to.
One of the planning place out of Manuel Antonio.
Which, oh my God, their Marino was just unbelievably beautiful, the ships that there were just ridiculous. Costa Rica is known for fishing. Your father had told us all about that before we got there.
They’re quite fishing community there, yeah.
Unbelievable and the food was awesome. Highly recommend anyone wanting to get away someplace close and easy. The costs were inexpensive to fly there. The food was exceptional. The seafood was unbelievable. The hospitality and the people were just so warm and embracing.
We had cops stop us on the street and say, policia here.
The police are here, my friends.
I was blind and they wanted to let them know this–
Wasn’t that cool?
It was nice.
And I felt that everybody was just very accommodating, appreciating my blindness for those–
It is very odd that the people in Costa Rica saw your blindness like people in the US don’t. It’s really weird how that is.
You’ve seen that. They all said they’ve seen me in New York and they avoid me like–
Everyone immediately was like, that guy’s mine.
We had a good time.
There was rainforest.
There were birds and monkeys.
Everyone speaks Spanish, they’re not English.
And John’s new nickname is Vikingo, because they thought he looked like a Viking. Yeah, anyway. It was a wonderful place, a wonderful time, and sorely needed. Things haven’t changed here in the States at all. We still have a lot of discourse going on. We still have a lot of dysfunction, our government is what it is and. The virus is waning or it’s ramping up. I can’t tell which and of the spectrum are on. But I can tell you that in Costa Rica, there were only 55 hospitalizations in 2021. So they’re doing something right down there. But again, we want you to be safe and healthy. We want you to wash your hands.
Yeah, I did notice one thing in Costa Rica. The people they were very fit.
We met people–
Might have had a play in the reason why they didn’t have so many.
And there will be a new blog coming out about disabilities that most people didn’t even know existed. We call it situational disabilities. Maybe you wouldn’t have thought about it as a disability, but there are people out there for a brief moment in time, who can say they joined the disability community and then got a invitation right out after they got better. But please read our blog located at myblindspot.org.
Feel free to check us out on our social media platforms LinkedIn, Facebook, and Twitter. And you can email us directly at podcast@myblindspot.org. And without further ado, here’s Dr. JR. Rizzo.
Hey, JR. How are you today?
I’m doing great. Thanks so much for having me Albert, how are you?
It’s a pleasure to make your virtual acquaintance JR.
Same here.
And if you wouldn’t mind, tell us who is Dr. JR. Rizzo.
Sure. I’m a physician scientist at NYU School of Medicine. I also hold a couple of appointments at Tandon and School of Engineering. Our research focuses primarily within two different scientific domains. The first is we work within assistive technologies, trying to break access barriers for persons with disability. And we also work within the realm of motor control. More specifically, we focus on oculomotor control and what that means for the limb. So eye control meets limb control or eye-hand coordination.
When you talk about eye, limb coordination, how does that differ for a layman like myself for some of the advancements we’re realizing for people with mobility issues being able to use their brains thoughts to move things?
You can still visualize without your sight.
Yeah?
Yeah.
OK.
Not that I’m the doctor here, but brief my understanding of it is that you visualize it and then you can move it.
See the one thing you and JR. have in common is you’re both good looking, but he’s smart. So JR. Help us understand more about how these–
Yeah, sure. So let me give you a kind of a quick thumbnail sketch of the world of eye hand coordination. So if we both in– I know at least Albert and I, we’re both visually impaired, although, we have different versions of this impairment and we could talk about that later and how they compare to one another. If you had intact visual perception and no problems with your visual system, and you did not have any problems with your motor system, eye-hand and coordination would work seamlessly.
And let me use an analogy, assuming your monotonically intact. So I put a Gatorade bottle in front of you, somewhere on your tabletop, maybe it’s about 15 inches off to your right workspace. And then now tell you to grab that bottle of Gatorade and take a sip as quickly as possible. So what would happen is, you would fire an eye movement with your gaze. Probably be looking at a specific location, probably the person talking to you at the time. You would actually take your gaze and you would point it, central vision directly at that Gatorade bottle.
Now, at the same time you’re doing that, you would start to plan the limb movement. Because we’d want to be extremely efficient in how we did this, and we’re very efficient creatures. And just in general from motor control standpoint, we tend to be very efficient when we speak of motor control in general. So I would start to plan the limb movement itself, and we would actually start to execute a limb movement. It would start in my proximal arm, my shoulder, and I would actually start to reach towards that initial Gatorade bottle based on what I was able to plan in peripheral vision or what we call the eccentric gaze.
And then what happens is, I said about an 18 inch reach or so. So that may be about a 10 degree second, which is a medium sized eye movement. You’re going to be able to do that in about a 20th of a second. Whereas a limb movement to get from an arm sitting on your lap to the Gatorade bottle on the table may take about a quarter of a second. So a much different time scales right.
So your eye lands on target, you now have this beautiful kind of central vision. We call that the 2K vision or two 2K TV of your visual system. And you can then refine the motor plan. And so you actually update the motor plan of the limb, and then you can grab that Gatorade bottle, grasp it, and then bring it to your lips for a nice sip of that Gatorade of your favorite flavor variety.
Now, what’s interesting about that is you might say, well, it’s somewhat trivial. I mean, I reach for bottles all the time. But what if the Gatorade bottle was really, really cold and it was a really humid day and it was speeding these big fat beads of moisture and it was slippery? Well, you may start to take that into account and slightly adapt your hand posture. And how you do that may be secondary to where the moisture is most intense. And that would all be appreciated given the eyesight.
Or let’s just come up with another funny analogy, which would have more relevance for gardening. But what if I put thorns or something sharp on that Gatorade bottle and you wanted to alter the actual thumb position of your grasp or even your index finger position? That would be appreciated with your central vision.
And so if I were to take the movement of the eye and I were to look at what we call the time series of the eye position itself and then I were to look at the spatial position of the limb and look at the time series of that, and I co-registered them, meaning I started the stopwatches at the same time. What we do in the lab is, we try and match those actual time series and co-register them, and then we look at what actually happens in terms of the spatial goal. How close are you to that shared spatial goal, because we know that it’s the Gatorade bottle?
But it’s not just the Gatorade bottle. It’s also what you’re grabbing the eggs to make the cake or the eggs to make your omelet or it’s the spatula in order to make sure that you can flip your eggs or where are you looking in the frying pan. All of these things are constantly this interplay between oculomotor control. And what we’d say is appendicular limb control. And not just that, but also postural control. And so how do these affect your systems as we refer to them coordinate with one another?
And what’s amazing is it’s really this beautiful symphony. Your body is constantly playing out this symphony. And as we know if we listen to music one instrument is slightly off, it sounds quite awful as we have one of the systems that start to get slightly off, you can cause a big, big problem. And what we think is that this could really impede recovery. And injuries with both either the visual and/or the motor system.
What’s actually happening is even though the motor ability of the systems themselves are OK, the functional recovery has really not come back to where it should be, because there are these missteps in the coordination and cooperation between these infected systems. So what we characterize in the lab and stroke, for example.
And some stroke individuals is not just– even if you have a pretty good visual system that’s now recovered or even that never had any problems, but you have some subtle motor deficits, what’s actually happening is you have this breakdown in this coordination between the two systems. And they’re not talking to one another. So that refinement of the motor plan that I talked about before with that Gatorade bottle reaches broken and we call this dicoordination. And that dicoordination holds you back and you respond to that and kind of funny way. So one thing that we’ve written a bunch of papers on is you end up making a whole bunch of extra eye movements.
So instead of like naturally I throw a bottle in front of you, I put a pencil in front of you, you may look and reach. And then you do whatever you need to do. It’s a look going to reach and then you may complete your task. And instead of doing that, what ends up happening, you may look eight times before you actually make that reach, six times before you actually end up reaching to grab that pencil. And you can think about how fatiguing and exhausting that would be, and how inefficient that process is, and what that would mean for someone with a stroke trying to recover or trying to seek gainful employment.
And so all these things become compound problems and one thing leads to another. And unfortunately, we really don’t fully understand these processes yet, and we’re still in the early, early days of how well this is going to get put together. But we’re excited, because we’ve created new routines and new procedures and we have new rigs. And we’re also now trying to create new platforms that are both on tablets, and creating new opportunities for clinicians to be involved across the care spectrum. And potentially, even in a home setting. So we can assess longitudinally right after the acute injury what’s happening. So hopefully I didn’t go too far down the rabbit hole there, but I think I was talking about–
I’m reaching for my carrot right now.
I was going to say, that’s pretty deep, doc. And I’ve got to say, I’m a huge nerd. So the way you explain that was pretty elegant, and I was following all the way up to the end there, to be honest. And while you were doing that, I was reaching for my drink and grabbing my drink and thinking about all the time. So I was tripping pretty hard Thanks for that.
So I’m not a nerd, I just play one on television. I don’t understand anything you just said, and I need you to break it down into laymen. So I was going but what does this have to do with being blind? And then I tie into the fact that we have that visual acuity to make that decision on how to move our body.
If I may, I think he was saying something along the lines of that. There is so many processes. That go on between you looking at something and picking it up that if even one of them goes even slightly awry, like he was saying with the musical instruments, if one goes awry, you can hear it. And not glaringly loud. If one thing goes wrong, it ruins the whole thing. And it can impede recovery from injuries.
So then JR. when we’re talking–
[INAUDIBLE] right?
- Yeah, I know he’s so smart. When we take these factors when we take these static things we know about how eye, brain mobility things happen in our head. Where is there the connection being made with stroke survivors, I get that. But with the things that you’re working on at NYU and the various inventions that you are creating to supplement or complement ability, where would we take that understanding that you’ve just described in instances for my own self concerns. I have no eyesight, so it impacts my mobility. But as long as I have a clue as to where something is, I should be going slower. You know Italians and our hands, it never stops.
Yeah, absolutely. So it’s a great question, Albert. So I would say that let me respond to the following way. I think Jonathan hit the nail on the head with his recapitulation. And to break it down for you. If we have a stroke patient, very concretely, we’re often familiar with seeing a stroke patient that has some subtle motor deficits or even some very severe motor deficits, and we can understand how that would affect eye-hand coordination very concretely.
And maybe I now took you on a little bit of a scientific journey and you see how that could hold back recovery. Same thing is true for someone with a visual impairment. So you can imagine now missing part of your visual field or being severely visually impaired. And what if you’re not taking that visual information or that sensory system is now some what impaired or damaged? What does that now mean for the motor planning system on the limb side? We just don’t know.
And so when we talk about this in terms of preferred retinal loci width. So you have someone with an age related macular degeneration. So central vision is gone. And they try and use a different part of the eye to visualize what’s going on. Now, what does that mean for eye-hand coordination? We’re just starting to unpack some of that, but we really don’t get what happens in this kind of coordination breakdown, which is how we’re really looking at it. And so these are all new emerging frontiers and we’re not really there yet. But we do think they’ll have a concrete effect.
Now of course, visual impairment has a full spectrum. 80% of people as you know Albert, who are visually impaired and legally blind have some residual vision. So what does that residual vision mean for motor control? We’re very interested in that. And if you have an impairment, how does that affect coordination process? It’s a different situation if there’s no light perception. And that then means what’s interesting is if you are using some type of supplementary system, some sensory substitute system in order to then reach, how does that then affect coordination?
And I think that’s a whole new emerging area and research that we need to spend time understanding and unpacking and we’re not there yet. I mean, there’s probably 10, 20 years of research easy to kind of unpack some of these answers. But I can tell you, we’re really excited because a couple of the things and even these early days that we’re seeing is– and again, this is all anecdotal and this has not been published or take all of it with a grain of salt.
But just by restructuring the eye-hand coordination and making these tasks simpler and I can get into what that means later. But I do want to start talking about some of the assistive technologies is that, we’ve seen some really tangible results pretty immediately. So what we do is we try and simplify the process, because we understand if the stroke patient is taking it upon themselves to make this task more difficult by taking in additional sensory information because that coordination process is broken down.
And what I mean by that is eight eye movements right instead of one eye moving, which we know is very abnormal. What if I then break this process down into something that’s easier for them to accomplish? When they do that, they find very, very positive effects, where they’re able to improve and their accuracy of reaching within very short time periods. And so you know this is early days and we’re really excited about it.
- hold on just one second, because I am so not used to not interrupting people. But you just keep going on and it’s a wealth of information.
He’s got a big brain with a lot of stuff in it.
I’ve got a little brain with not much in it, except a desire for a beer right now. But one of the things that happened to me and it’s odd that this happened just prior to us getting together with you. And it’s going to go back to the eye-hand coordination bit. John and I were out at a restaurant at a bar, and I kept picking up the water glass as if it was the beer bottle and putting it to my mouth to look for what I knew was–
And it was a specific glass and had the same diameter as the beer bottle. So when you put your hand on it, it felt like the beer bottle.
Yeah, and it kept confusing me that I really had to ask for a different glass, because my brain told me you’re touching a beer bottle, and you need to tip it a certain way and wait for the little openings instead of the big opening. So right there, the way you were just describing it, my little brain was able to grasp that I need to figure this out, so I can drink beer better.
But anything we could do to facilitate beer drinking.
Pour some good wine Dr. Rizzo, because you–
Definitely write that in our proposals. That’s one of our key takeaways.
And when we do that together, you put the NIH publishing. And because blind people need to drink beer without dropping an ounce. That’s perfect.
So all of this research is– so it’s all new and groundbreaking. One of the things that–
I also want to point out, it’s probably very obvious, but we’re talking about the central nervous system, which is amazing like ridiculously complex. And he was saying earlier that they’re just getting started on figuring out what even it does. And this field is very, well, unexplored.
That’s one of the things that gets me excited. And one of the reasons that JR. and I are just [INAUDIBLE]. It’s the universal solutions we’re talking. We’re not just talking about how it impacts blind people. The fact that people with stroke–
It should be close to your, I would say heart, but I guess head because it’s your optic nerve, which is a very big piece of the nervous system and very, very important use of the nervous system.
That’s why certain opportunities or certain breakthroughs are not going to be as promising for somebody with the way I lost my eyesight versus–
Trusting your neuropathy.
My neuropathy.
Yeah, and one thing that’s interesting, real quick, Albert, is it’s interesting what’s– I mean, I’d be curious to hear more of your sensory experiences, because what’s amazing about the brain is this concept of mental modal plasticity. And there’s so much vision real estate in the brain that focuses on processing information from the eyes. And when that changes– and now I can tell you in a situation like yours versus mine, I can be very concrete and open about my transparent. I’m probably at about 1/13 or 1/14.
So 1 over 14 of what a normal person sees. So I have a very small amount of– so I’m way under utilizing my real estate. So if I had 14 acres to build on, I’m only working with one acre. That’s the way to think about it in terms of real estate. So you’re nuts because you could build a mansion, right?
Well, that’s true and I have a very little amount left. Now, I have a whole lot of brain than I could do a lot of really interesting things. And the brain being able to adapt in terms of mental modal plasticity, I’m wondering if you’re taking in other sensory information like proprioception. So that kind of joint position sense and other things. And the fact that you were used to grabbing a vessel a drinking vessel of a specific size and caliber, you were then able to say, hey, listen proprioceptive this is telling me X, and from motor control, this is what I expect. And that’s been a procedural memory and what’s actually happening.
One interesting thing that I’ll share with you and you may know. I know that you met him at least on our scientific advisory board. Dr. Lotfi Merabet, from Harvard has done some amazing research. And he and I got so excited because we wanted to keep this low key. But there’s a lot of beautiful work with cortical visual impairment in children who have cortical visual impairment. And we started talking about eye-hand coordination, he was blown away because the type of stuff that we’ve seen we’ve done more work, I would say, and acquired brain injury.
So that’s basically looking in and seeing what happens when you injure the limb motor system in a lot of the patients that we bring in. What happens to the coordination? But he was saying, I see a lot of what happens to the vision system. Particularly, you damage the cortex visually. And then what happens. And he was saying that a lot of cases when he sees what would be eye-hand coordination with these kids they have very unique adaptive strategies to do that, and it’s almost like they try and limit the visual information, which would make a whole lot of sense between some of what we’ve been seeing in terms of quieting the eye down.
And maybe the last thing they’ll share on this, which I think is really provocative is the sports world and sports medicine world is filled with elite vision training. How do you become the next best Olympic athlete? One of the emerging areas in this field is what’s called quiet eye therapy. So a lot of people probably think if you’re going to be the best athlete in baseball or basketball, you probably have really great eye movements and you’re a fast eye moving maker and you’re just an amazing in terms of motor performance.
So a lot of that is true but let me tell you what’s untrue. You don’t make a whole lot of extra eye movements when you’re a superior athlete. The well-trained athlete at the elite level actually makes less eye movements and those–
More efficient, yeah.
–their eyes. So they’re much more efficient in that process and they pick these anchor and key target locations. And some of that probably has to do with what’s involved cognitively in those processes, and how they can then answer the questions methodically. And so it’s really interesting, but we have a whole lot more to understand. But I should probably leave it there for now.
It blows my mind. Because now as I’m digging through my memories, I have a repository full of visual references down to colors and everything. John and I are just redoing our entire house, and I get really granular about color as somebody who is severely blind. I have no usable vision whatsoever, and I get really caught up on the details, the textures, the lines, the colors.
In my opinion is just paint everything black.
Yeah, but he goes with his soul. But I’m also thinking about the time that I worked with the brain port, which was at its time with a great concept. But realistically, I wasn’t going to be walking around with the depressor in my mouth that’s going to be sending electrical impulses to my tongue. But when I was able to have things going back to the visual acuity with regard to mobility options for moving closer to the object and grasping it.
When I was able to perceive, I won’t say see, because it’s just not what it was. Perceive the shape that was in front of me on being beat out with electrical impulses on my tongue. I was able with the way my brain processed, that black on black image, oddly enough. I was able to also guesstimate the distance I had to go to pick up by the size of the image that came across on my tongue. This is really–
That’s what I was saying about visualization. You were having some kind of input on your tongue. So your brain was using different parts of itself to make you see like you said black on black image or something, that’s like a visualization and you were able to more easily reach and grab that.
When you said that, that made me think of that experience years ago.
Yeah, and if I can go back to the sports science reference, I’m a sports science guy myself. And that was a beautiful analogy of how an elite athletes use less eye movement to track, say, a ball going through the air or something because they’re more efficient at it.
And proficient.
Yeah, it gives great hope for someone who doesn’t have that eye movement, that they can work with it like an elite athlete, it’s crazy.
The fact that it’s doable means it can be replicated in some way by sight.
Not only that’s doable, but that it is more efficient with less eye movements.
Just quickly to finish on those points that you made, which are all really good ones. The key in a lot of cases is these elite athletes, they become amazing anticipators. They don’t have to track the ball speeds, they already know where they’re going. They don’t have to figure out what’s going to happen with the pitch, because just based on the shoulder movement of the pitcher, they know what type of pitch is going to be thrown and where the ball is going to go.
It’s a lot of intuition.
They build all this– they have this amazing visual information, because they then give themselves these enhanced time windows to respond rhetorically. And then the motor responses are beautiful. Because these guys are just lined up right and they know exactly what’s going to happen way in advance, and that’s what’s amazing about what’s happening. So can we train that anticipation and that’s–
Of course we can. Something like that comes with practice, repetition, doing it over and over again. The reason why a batter knows to start swinging his bat before the ball is even left the pitcher’s hand is because he’s done it a million times.
How did you? What came first, the chicken or the egg? Did you start studying all of this because of your personal connection with the visual impairment or was there something that you as a child woke up with that made you a major nerd geek scientist who was thinking about this his whole life?
No, I mean I literally tripped into it. I always like to say that I have my doctorate in medicine and my PhD and tripping. So listen, at the end of the day, I would say that I was a geek and a science nerd. But I didn’t know what I wanted to do. And to be honest with you, I thought I’d go into ophthalmology. But I had some hard conversations with people and they said it would really never be possible. And I have some regrets because a lot of people have told me no, I’m sure Albert they’ve a lot of people along the way.
And I think you’ve seen in the bottom of my email address, my signature line says, the person who says it cannot be done needs to get out of the way the person who’s doing it.
Yeah, exactly. There’s a lot of great quotes from Helen Keller that gets over. “The world is full of adversity and also full of those who are overcoming it right.” It’s a favorite quote from Helen Keller.
That’s a beautiful one.
I didn’t really know what I was going to do, and I knew I wanted to go into something. And then when I got this diagnosis, it had to be vision related. And to be honest with you, is just necessity is the mother of all invention. And I just knew I needed to work on vision sciences and I grabbed whatever experiences I could, and I tried to pay it forward. And then a lot of it was just trying to squeeze as much juice out of whatever fruit was available.
And that’s really been the story of my life from day one. I really didn’t have any preconceived notions. I knew that I needed to go into vision sciences. And once I got diagnosed with this after about a year of depression, going back. So I think you’re asking about the story. So I was diagnosed as a young teen. I never knew what it was like to see through normal eyes. In well-lit conditions, I’ve always done pretty well. I’ve had severe tunnel vision progression over my life.
But I had terrible nyctalopia from a young age, but I never knew what normal was like. So my mom always noticed the abnormal behavior and mentioned it to a local optometrist, but he didn’t know what to make of it. And he kept testing–
- stop there for one minute. I think you and I are going to say the same thing, John. nyctalopia.
It sounds like another word I know. Nyctophobia, which is fear of the dark. So I’m assuming that nycto is the dark.
And then missing. So missing, I don’t have night blindness, basically.
[INTERPOSING VOICES]
You were diagnosed with this as a teenager.
My childhood was like the following. I would go to camp or go to school and I would do OK. And then I was a nerd. I would do all my homework and then my brother and I would go outside, and back then this is 40 years ago. He’d be like playing the mud and trees and whatever, and it get dark out and we play, manhunt or whatever. And I couldn’t do it. I would always be at home base and I could never see anything. And I never knew what to make of it. So I wasn’t sure what to do and how to respond.
And the other thing that was very glaring to my mother is whenever we went to– I mean, my father was a workaholic. I never really saw him. But the one time I would see him is– yeah, Saturday night we’d occasionally go for a burger and a movie, that’d be our family time intermittently. Whenever, I got in a movie theater, I’ve never been able to see in a movie theater, like nothing. As soon as I go in a movie theater, I just don’t see anything, even if it’s like dimly lit. Everyone’s like mulling around and finding their seats, I could never see anything.
And so my mom was always notice the fact that my brother would run around and pick the best seats, and I was always bumping around and I would try and come up with some strategies to figure out how to do it. But you know it was always very odd and she noticed it. And the same was true when we like go out at night or whatever. I tended to never go outside like boy scout activities, I’d always wanted to be inside by the lights.
So anyway, so the local optometrist pushed it down and kept testing me in well-lit environments and I would do OK. And then finally after about four or five years, literally of saying the same thing, my mom just got aggressive enough where she’s like please do something, like some testing. And so he did what called a parametric evaluation, like a peripheral field test basically. And I got out of it and he goes, you must have had a long night studying or something, you fell asleep. And I said, no, I didn’t. And he goes, come back tomorrow.
And I said, all right something’s wrong. So I said– that was the start of the roller coaster ride. So it came back scored the same the next day, and he’s like, you’ve got something seriously wrong. Got to go see some diagnostician, then I thought I had RPP. And then for about a year, I had every eye test. Had about 35 parametric evaluations, multiple Eris, they did like everything and they weren’t sure what. They was they thought maybe RP, and then I ended up with choroideremia, which is a sister condition to RP, but slightly different. That was diagnosed at about 12.
At the age of 12, wow.
12 or 13, yeah.
And it’s interesting because–
Very pinnacle point in your life.
Yeah, and we talked to a number of our colleagues and peers in the industry that I know you know. But each of them who had grown up with this had different experiences in academia and elementary school, high school, college. Were you put in special ed because all of a sudden you learn differently and they thought you had a problem as opposed to the teacher’s not being able to teach to your ability?
Not as much. I mean, I remember having some issues where I’d asked the teacher to sit-in the front. Again, in the well-lit conditions I would do OK and I was a severe myope. And I had some issues. But with the right refractive errors and I could optimize my system. So I would feel pretty good, like I was pretty good about trying to come up with ways to do it. But I mean, you’re a kid, you’re wearing thick glasses, you’re sitting in the front of the room. I mean, I got razzed a lot as a kid. So I mean, 13 to 18 was not a fun age for me.
I feel you. It was great for me.
[INTERPOSING VOICES]
You accused him of what. I had to go to resource and special ed–
I think, I asked the question. Don’t say it the way. John, has dyslexia and was always sent to the resource room since–
I’m slow.
It seems you’re not so. It just seems that that’s always the first and so our first line of defense for educators in elementary school and middle school to send them to special ed.
I don’t think what he had would be considered a learning disability.
That’s why I want–
It’s again just like OK, you can’t see, moves in front of a class. Can you see now? Yes. OK good, done.
Yeah, but still he was using one of his weakest muscles and he still has a PhD. So it didn’t adversely impact your academic enrichment, did it?
Obviously, he’s a doctor.
No, I mean not. No, I wouldn’t say it did.
Acquiring the knowledge is a pain in the ass in plain English when you have a visual impairment or a disability.
Yeah, I mean I tried to. But to be honest with you, the rule of thumb that I followed, which is not the one that I advocate for now is to shut up and accommodate and figure out how to handle myself and–
I agree with that.
The right way to deal with it because I didn’t want to create problems. And this whole concept of labeling. And still to this day if I look back over the last 20 years, I could tell you it’s over 50%. But let’s just say the majority of the time people say, you may not want to lead with the fact that you have this visual disability. And I disagree, mean, I’m at a point in my life where I think it’s important to lead with it. Because we have to flip the script and understand that in certain situations, there’s two ways to look at that. I mean, the easy way to look at it is disability is a weakness, that’s what a lot of the initial knee jerk reaction reflexive response is.
I don’t agree with that either.
But it’s total BS and we all know that for those who thrive in the environment. Disability gives you profound strength.
Yeah, absolutely.
I’m a strong believer that within your disability, you can find your strength.
Well, in any ability. It’s just that plain and simple. But the fact that you said something and John agreed with it too, and I never had to deal with it, fortunately. Is that we don’t fight or argue for ourselves that way because it’s best for us not to admit our disability. And that’s only been a recent change to be able to be loud and proud about your disability or at least being clear eyed about how you need your environments to be set up or how you need your classroom environments to not so much accommodate.
But just to acknowledge your ability and go from there. All too often, we wind up– assuming Johnny can’t read because Johnny’s either stupid or just can’t win and has nothing to do with that, has to do with the situations or the circumstances that each of us as learners bring to the table. And our society as a whole, not wanting to acknowledge. And we’re running into issues with that in school today, as well as work environments.
Because people don’t self-identify as having a disability, just like they didn’t self-identify as being gay years ago. So there’s been some advancements there, and I think there’s going to be an opportunity. But I was just curious because when you were saying to me about how you really couldn’t see at night, I was curious about what impact that may have had during the school year or school day, and it doesn’t seem to have had all that much except for you getting made fun of because you sat next to the teacher. Teacher’s pet.
Well, I mean, there were other things. I’m just saying largely, I would say my education. I mean, that was during college and things. I had to change a lot of things. I mean, again overall, I would say I’m privileged to have the fact that I was still able to use a lot of printed content. But with some modifications, I was able to use a computer. I could change font and contrast, if I wanted to. My brother used to joke with me. He used to say, what I did in college and what carried me through med school was what I created what he affectionately called the nerd pod.
So I used to have multiple monitors with bigger screens up. And I used to create usually three desks so I would have typical computer desks that would an L-shaped computer desk. And I put another one on either side. I would sit in the middle and I’d sometimes have two or three screens with textbooks and desk lamps that would provide better ambient light.
Nowadays that’s just a really cool gaming rig, to be honest.
Yeah.
I’m planning on building one like that myself.
And I’ve been doing that for 25 plus years. Building nerd pods. And now some of that stuff is harder and I have to do different things. But there were a lot of things. I mean, it’s just it depends on what you want to talk about. I mean, if there was activities at night I couldn’t participate or if I wanted to drive it was hard or I had to hitch rides. When I went to med school or school, one of the reasons influence where I needed to go is because I always had to use public transportation or hitch rides. Everything was turned on its side. I mean, there was a lot of adaptation.
I mean, this is we talk about with strength is your whole life is an adaptation. I like to say, the analogy I give students when I talk to them or people who reach out and say, who are upset because they get a new diagnosis, I said, what you don’t see right now is the strength you’re going to gain. And it’s things because you’re in this low point. But you need to step forward you’re going to continue to do really well. And life’s going to be upside down, and that’s never going to change.
But living life upside down has a lot of profound upsides. Because all of a sudden you’re in all these other environments, whether it’s work, professional, you have all these amazing skills that you’ve just developed. Because everything becomes somewhat difficult. And you all of a sudden have this confidence and you have this sand floor whenever you complete all these things. And you get that much better, whether it’s public speaking or needing to stay calm and meetings or having more control over yourself when people are losing sense of themselves.
You have all of these great skills that you’ve now built up and diversified over the years, which were just secondary to getting in challenging situations because of your disability. And you get all of this stuff. You get this super PhD for free. And so you don’t see it right now, but you are getting this amazing education. And you’re going to be able to pay that forward you’re going to be awesome expert no matter what happens. So keep taking one step forward every day. And if you do that, you’re going to be doing yourself a great service.
And you take one step forward every day acknowledging your successes. And in some way, acknowledging your defeats and just keep on trying. Rome wasn’t built in a day. Edison always said, he got it right once. But it’s the thousands of times you got it wrong that led him to the success. You touched on something too before that I think is important for all of us to appreciate in life in general. You mentioned how once you were diagnosed and you all understood what the problem, you went through a depression. I appreciate you sharing that. And bidding to that. And I think we all have to learn from that and say, OK yeah, we do get depressed.
You’re not going to be able to– I mean, being blind is not for the faint of heart. I would probably say that friends of mine who were deaf would say the same thing. And people in wheelchairs would say the same thing. But we’re a resilient species. We’re very adaptable, malleable, and pliable and flexible. And I think if we can adapt, we’ve just got to believe that it’s OK to be angry. It’s OK to be depressed, but it’s not OK to wallow in the self-pity that holds you back from not becoming a PhD or lawyer. There’s nothing you can’t do.
It’s just like I’m on my–
at my Spot, we believe that access to the right tools promotes our own ability, and thereby restores infinite possibilities that we ourselves can create.
Absolutely, it’s a beautiful message. And there’s a couple of different axes you need to appreciate on disability. One is, it visible or invisible and is it static or is it progressive. And I think that means different things in terms of the overall psychology of what’s happening. And you have to be careful about that is it needs to be appreciated and it’s complex and it needs to be handled in specific ways.
We give an intro to disability lecture to med students every year. And I try and make these points and I try and say, it’s important to make sure you understand about what’s happening and the physiology behind what’s going to occur and what’s going to happen next. In a lot of cases for many, many years, I was told and what was very frustrating for me is I kept being told, make preparations but there’s nothing can be done. And to be very concrete with you, there is still nothing that could be done for me. And that’s very frustrating, is to be in 2021 and have access to the most amazing medical world and still there’s nothing to be done. It feels very frustrating to be in that situation.
But I can tell you one thing one thing. That I’ve learned over a lot of years is what feels incredibly good is that I like to think of building this amazing team. So Albert and I, look at it we’re on the same team. We’re on the VI team, we’re on the Visually Impaired team. At the end of the day, what makes me feel happy and motivated and willing to want to move forward is constantly thinking about the kids who are getting diagnosed with the conditions and the feelings that I felt earlier on.
And having some conversations with children in age ranges like that and being able to think about how we’re going to work really damn hard to make sure that stops. And that type of engagement and that motivation and that drive, there’s nothing stronger than that. And one of my favorite things to tell my sons is, the fire in your belly is always stronger than the fire in your mouth.
And there’s something to be said for the fire in your belly that bruise and storms when you get this card. It’s a shitty card to be dealt but when you understand that you’re getting put on this team, that team is always going to pull for you. They’re always going to root for you, and you’re always going to try and make it better. And there’s a certain kind of legion of individuals that are out there that I continue working no matter what. This isn’t a game. This isn’t a business, this is about trying to make it better. And you’re going to be able to count on them no matter what, because this is what’s mission critical to them. There is no other option. It’s option A, A, A, A, in multiple choice. So once you understand that, that becomes an important message, I think.
And that’s one thing that has endeared me to you and allowed me to call you not just a colleague, but a friend, is that we both agree about the concept of a team. I saw yours in action. And I think you’ll agree, team, is an acronym for together everyone achieves more and that’s where you and I both want to help people who come behind us and are introduced to the blind community or the greater disability community, that there’s a team of us there who can help you achieve more. With the card you’re dealt. I think that’s a very important concept that has become more crystal clear to me as a blind person than it ever had been before.
That’s why I think a lot of what we do at My Blind Spot, is more for I would say, preventative medicine as opposed to triage. I’m looking for the people like us who have transitioned into the disability community or blindness community after having had vision for x number of years or which doesn’t happen as often as it used to, but it still does happen. People being born into the community Thanks to advancements in prenatal care and health considerations and advancements in that arena, we’re able to stave off people being born or made blind by incubators. So I think the younger generation and I’m talking younger than like 40, these days even younger than 50 depending. Who have been born into the community. They want to do everything on equal footing.
That’s absolutely true. The interesting thing is in a lot of cases, we’re in for a bit of a shock and when most people don’t realize is we are doing generally well. I mean, pandemic aside which has been awful. Medicine is amazing. S We spend 20% of GDP. I would argue we should continue spending 20% on GDP. We should figure out how to better find it, but I’m all in favor of having an economy that really focuses on health care to keep our populations strong and as healthy as possible.
And we’re increasing life expectancy. We’re doing really well. In certain cases with some cardiovascular diseases. But what we don’t appreciate is what’s coming down the pike in terms of visual impairment. And you made a really good point when you said, these younger kids. I mean, we’re also doing that in neonatal care and obstetrics. And retinopathy of prematurity is a big deal. And I’ve given some talks and I’ve had some neonatologist come up and they’ve said, I’m really excited to hear that there’s some assistive tech stuff coming. We have kids that are coming and they’re really young, but we’ve been doing more now to save the lives of these younger kids. When unfortunately, they’re going to have a life filled with visual disability. And we need to be able to better support them.
The fact of the matter is in any civilized society in any modern country that’s out of the third world, forget about low and middle income countries, s the numbers are still the same as they have been for decades. You know this it’s like 70% to 80% unemployed, maybe 20% to 25% employed. I mean the numbers, we need to flip those numbers. [INAUDIBLE] absolutely and I think it’s refreshing to hear doctors.
And I always think that the best route for, at least, as I focus on the blind community and people with visual impairments talking to the doctors who are charged with saving our eyes. And when our eyesight is told yeah well it’s never going to get any better, it could probably get worse or you’re never going to see again, I’m making a generalization. And I’m sorry, but I’ve seen it happen. They go, OK, we can’t help you. You’re blind.
And there’s no support, there’s no transition. It’s not even on the eye doctor’s websites do they have assistive devices to introduce the concept of magnification or voice over technology. And I think what’s exciting is you’re hearing doctors appreciate that there are assistive devices there that can then be incorporated into our academic environments. So we can educate students of ability and teach to their ability, rather than having them use their weakest muscles to try and be, quote unquote, normal.
Yeah, absolutely.
That’s the promise of tomorrow that I really look forward to.
Some of these kids are going to be like JR. and have a vendetta against their disability and go after it hard.
And we need them in their nerd pods. I just got to fix–
They have the passion.
Now, here’s another thing too. I want to change it. Now we’re talking nerd pods, gaming and all of this stuff that John is probably in his head, thinking about. Last night he went out with a couple of friends and had gotten deep into a rabbit hole with virtual reality. I want to do that. I want to be Picard on the Holodeck in Star Trek.
We’ve explored a little bit.
They do make soundscape.
But we’re talking to Dr. Rizzo right now. And I want to know. As a person who’s blind, where do I start looking at that being transitioned to people who have no visual acuity at all?
Well, it’s perception. It’s interesting. I mean, we’ll see. I mean, we’re talking now about brain computer interfaces and BCI technology. So there are really seven academic centers that have what I would consider very significant BCI programs. And they’ve grown substantially. And they’re doing really amazing things these days. And I feel confident that we’re going to see a lot more in the upcoming decade as it relates to brain computer interfaces and we may be able to see some really interesting things when it comes down to some type of immersive display or s taking those inputs and just going directly into the visual cortex in a specific way.
I don’t know, I mean I think we’ll see a lot of tremendous progress. The jury is out in terms of what we’re going see. I mean, the central nervous system is so tough. And it’s just interesting. The body’s natural response to foreign is to reject. And so they’ve done a lot of amazing things with these electrode arrays and creating multiple microarrays. And motorically, I don’t know if you’ve seen some of the work out of Pittsburgh.
But there’s even a 60 Minutes special. You can Google it even with Mike Berninger in 60 minutes, hear a little bit about what they’re doing there. But they’ve had some spinal cord injury patients with brain computer interface is reaching with robotic control. It’s amazing and could that apply to the visual cortex.
But the thing right now is what they’re trying to figure out is what are they going to be able to do in terms of minimizing neurosurgery to get those implants in? Because the body rejects them after a certain time period. And so you’re buying yourself surgery after surgery, and that’s just untenable. You can’t imagine yourself having surgery every 11 or 12 months to implant new electrodes.
While we’re on the subject, I would like to get your input on what do you think about Neuralink.
I think this the idea of let’s call Neuralink like less lead or lead light EEG technologies and other types of things. I mean, I think we’re going to a whole lot more in terms of EEG and new, I would, say non-invasive ways to map the brain.
And what EEG?
The electroencephalogram. So reading the brain spikes and activity, if you will. It’s like what the brain uses to communicate in between the cells.
Thank you, OK.
Sure. And we’re going to be able to do a whole lot more with what’s called signal processing engineering to unpack those mysteries. The jury’s out though, again, in terms of figuring out how are we going to use these approaches that are less invasive to get the more complex outputs. And the brain power and all these other things and a lot of these sensory substitution of devices are OK within a specific range.
Problem is, the visual system is magnificent. If you look at Nick Medici, who I believe you met on also a scientific advisory board, he has some great publications where he’s mentioned what are the bandwidth differences between vision and touch. Some people estimate in his publications, he’s defended 500 to 1. There’s a 500 to 1 bandwidth a difference. And when trying to estimate that. And the interesting thing–
That’s in favor of the visual–
Right.
Because I would assume the optic nerve is like a big drain, right?
Well, let’s just play the numbers game. I do this sometimes in lectures. So just for fun the retina itself in each eye is composed of approximately and this will depend on the post-mortem study and cadaveric studies, but about 100 million photoreceptors. And then those photoreceptors, you have this amazing kind of complex, these interactions between bipolar retinal ganglion cells. And those RJC cells then would end up going into the optic nerve into one big fiber bundle, like a big power line to the back of the brain.
Now, what’s amazing about the brain we have $100 billion neurons in the brain. And some people say, between 25% and 50% are focused on vision. But let’s just do some quick math. I have $100 million photoreceptors and then I have to get it down to one million axons in the optic nerve. So there’s 100 to 1 compression. And then, I need to upscale that later on to $100 billion neurons. And if we say 50% are focused on vision, that’s 50 billion neurons, and that’s not really a fair equation.
But you can imagine all of the complex interactions that need to happen. And what’s amazing about the eye itself is, the eye is already set up before it gets into the brain. It’s amazing ability to process edges and contrast. And so it has this concept of lateral inhibition baked in to the retina itself. So you’re able to do processing in the retina before you even get out of the retina.
So we have to figure all these things out, in order to then figure out how we’re going to get vision perception to equate to what someone would naturally see. I mean, vision is this amazing experience that is not like a TV where you just hit the power button and all of a sudden it works. It’s this amazing set of machinery with all of these crazy connections, and lateral connections. And it’s tuned over decades of development.
Stop there for a minute, JR. Tuned over decades for with regard to development of the file cabinet full of visual information.
Yeah, decades of research trying to figure out something that’s been like what–
Billions of years evolving into a human brain.
Yeah, but at the same time one of the things that from the blind community that’s very blind centric is, if you were born blind and you’ve never fed information to your visual cortex, how are you going to process that information? Then he’s talking about the receptors in the retina who then take it down and punch it into billions of other things and all the confusing things that I didn’t follow. I am wondering, will there be limitations with the technology we create once we figure out how to process that information?
Of course, there will be, correct?
Yes, definitely going to be limitations, depending on when in your developmental process you either lost the vision. Or first of all, if you’re generally blind versus blinded, what’s called adventitious blind, much different. And so we’re going to have to be strategic and tailored in our approaches. And then depending on where you are in development when you lost that vision, is also very interesting.
And there’s some amazing articles that have been written about this. If you actually take people who have congenital cataracts in India, in certain cases, in countries–
Huge situations in India cataracts.
The cataracts, right? And you give them vision like a 20-year-old. Some people say, I don’t want the vision. It’s so overwhelming for them to handle. They want it, because–
I’m sorry, because that way I can sound intelligent talking to the two of you. What normally happens is, and there have been documented instances. And again, India has some really unbelievable populations of people who are born with cataracts, acquire cataract, it’s a huge problem there. But when they have not processed information to fill their visual cortex with information, it becomes– Oh, remember when we were watching supernatural yesterday? It’s like, hold on a second. Watching– we’re not going to talk over each other, you taught me that. Supernatural, she could hear all the Angels talking and she was going crazy with the amount of information she was giving. It was distracting, it drove her crazy.
It can be so distracting to a person who had no visual acuity at all to be bombarded with information like that.
It’s like living your whole life in a movie theater and then stepping out in the sun, it’s blinding.
Yes.
I understand why some people might not like that, might want to go back, or might be upset about it or whatever. Personally, I feel like if I were in that situation, I’d be like, oh good, I can see better now. I’ll deal with the pain. It’s individual unto each person, and that’s one thing that if and when. My problem is, I think stem cell research is going to be one option for me when we get to the point that we can actually put stem cells into the optic nerve.
But I’m going to have to make sure that the doctor or the team that gives me my stem cell rejuvenation, make sure that my mother always looks 59 nine forever, because that’s one of the things she’s happy about me being blind. She’s always going to look 59 forever.
I’m banking on computer stuff.
Well, stem cells I mean I think are an interesting area but depends on what ends up happening and the damage. So let’s take a specific situation. So let’s say you have cortical visual impairment secondary to a stroke. You have damaged tissue. So the idea would be now I’m going to go into this damaged tissue, I’m going to put these stem cells. These healthy stem cells and they’re going to regrow into those neurons.
Maybe that would be true and we have some initial positive effects right now or stem cells could be helpful. But a lot of people think, unfortunately not to cut the hype, but the issue is they’re going to be a lot of humoral factors that it could be very good. Like things that may be in the actual stem cells, not just the cells themselves that may be beneficial in the microenvironment. And what we call the internal milieu that may be good for that situation. If I take damaged tissue–
Let me give you another analogy. It’s like saying, if I were to go down into my basement. And I would have like let’s say I lived in this big house and I had all my circuit breakers going in and I had like five circuit breakers lined up and two of my circuit breaker stopped working in the equivalent of the vision cortex, and this guy went down there and you said I have all these right. I have all the right breakers, everything’s going to be fine. I’m going to take these breakers, show up one day, I get a really good electrician. And rather than actually get all the connections connected and hook up the breaker boxes correctly, he just throws all the equipment in the basement. And he says, OK, everything’s good now.
We’re missing all the connections, right? All those connections haven’t been made between all the neurons that were in that damaged area from the good tissue, all the surrounding tissue. And we don’t know how to get those connections to then be rekindled. And that’s what we need to focus on. And I’m not sure where we are or what’s going to happen.
So I’m only walking through this because I want you to think about, also if you’re going to sign up for things, so you’re knowledgeable about the space to understand there may be aspects in the microenvironment that are good but what are they then speaking to in terms of rekindling those connections and what’s going to happen? s what is the data to suggest the fact that these connections are then going to be able to re-establish themselves correctly to then make sure that perception is sound and intact? Instead of making sure whatever I scratch my forehead, I started seeing your mom at the age of 80.
Now, I hear that the way stem cells work is it’s better if you get it like immediately after the injury. The closer to the injury that it happens, the more stem cells can do. Because the body wants to heal it just doesn’t have the ability to do it. Something like that, I don’t know.
Yes, when you’re closer to the injury as a matter of fact, in certain cases, s have some pretty amazing data. And this is an animal models, where you can re-stroke animal models. Like after you’ve had a stroke in order to improve recovery.
Which we would never think about doing to humans.
Like reverse stroke?
If you give people like you give a stroke again and they actually improve.
Well, JR. we’re coming to the end of our time here. And I just have one last question. And if you would just briefly tell us about what your plans are for the future, what you got going on.
Sure. So one of the thing that I think will be great to just touch on briefly is a little bit more about specifically the side of our research practice where we focus on advanced wearables. And I know that Albert mentioned that. Previously, we’ve been working on for about 10 years now what we call, different versions and embodiments and form factors and it’s earliest roots vision to the fourth or a visually impaired smart service system for spatial intelligence and onboard navigation.
And you can really think about this as a personal mobility solution that serves really as a customizable human in the loop, sensing the feedback platform to really deliver any type of functional assistance that you may need. And so the simple colloquial way to think about this is, driverless car meets a visionless or someone who’s a visually impaired pedestrian. And we’ve created these really neat backpacks that have miniaturized sensor systems baked right into the shoulder straps or in modified sternal straps, which you may use to alter the payload of that actual book bag.
And what that’s able to do is we can then digitize environments in 3D. And in real time, feed you back almost immediate information. Both through audio and through touch. So the way straps of that same book bag can give you tactile cues and prompts, and we can also be to different vocal prompts about what’s happening.
So you can interact with the platform, the platform are also getting geared up so we can anticipate what you’re going to need in the future. And it can really provide a whole host of microservices. So very concretely, it can help you with basic navigation and street crossing to identify pedestrian crossing signals.
So even in our very own New York City unfortunately, the majority, the overwhelming majority in fact, over 90% of pedestrian signals signalized intersections for pedestrians are inaccessible. And so can we use our sensors and largely computer vision to then identify red green or walk safely, and help give that information to the end user. Also, do things like understanding the intersection geometry, where you are in the crosswalk and give you some veering corrective cues to make sure that you maintain a safe path within the crosswalk and maintain a centered in the straight and arrow right in the center of those zebra stripes, if you will.
But then we could do all kinds of other things. Like we could support shopping assistance. So if we talk about other activities or instrumental activities of daily living as we like to think of in the rehabilitation medicine world, we can really unlock a lot of those services through this book bag. And so there’s a little processing unit that would get thrown in an inner sleeve. There’d be an area where you could just throw your groceries or books or what have you. It would look very commercial and streamlined, so it would be inconspicuous. And you would use this binaural bone conduction headset that would be tethered to it through Bluetooth.
Bone conduction?
Yes, so a bone conduction, if you’re not familiar with it, it’s just different than air conduction. So when we typically use spoken word, I make little vibratory waves in the air. Those go into the middle ear, the eardrum. There’s these little bones in the middle the middle ear that transmits a fluid wave actually in what’s called the cochlea. And then you get these little hair cells to depolarize.
You can actually use bone conduction these neat headsets that are actually being used all the time recreationally, either float over the bone behind the ear or the jaw right in front of the ear. And actually vibrate small waves directly to the cochlea. Now, what’s amazing is you can keep the ear canal open, which I think is a visually impaired persons number one defense mechanism.
Yes.
And also process the bone conduction for the information simultaneously. So you don’t lose any of those defensive mechanisms.
That’s so cool, man. So now, the information gets processed and it helps the individual move through the city, move through an environment. Is there any need? I mean, how does it collect the information? Is it through triangulation or is it through beacons? I mean, what’s the underlying technology?
Yeah, great question. So we use these sensor systems it’s largely. The thing of it is kind of triangulation and also what we call computer vision based and through disparity. And so just like humans can sense depth because of the fact that there are stereoscopic, we have two eyes. And their front facing. And so we’re constantly basically creating depth maps secondary to triangulation. We could do the same type of thing with the system.
And so the idea here is, how much can we really do with this actual powerful book bag without depending on local infrastructure? So even though we have some research for us where we are looking at doing Wi-Fi fingerprinting and looking at local infrastructure because it would be silly for us not to take advantage of that and things like airports and moles and larger transportation hubs. Can we not try and create an independent platform that would really be able to for most intents and purposes be what we consider infrastructure free. Not needing some type of a dependency in the actual building itself.
Now, I will caution that some people consider mapping and we do need maps in order to function. Some type of a pre-installed information set or an asset. And some people consider that infrastructure. I think it’s somewhat different because those could be downloaded at your home, depending on your trips. And you don’t necessarily need to worry about building outfits or really a lot of construction needs or a lot of permits.
I mean in most cases, the type of mapping that we do can be done with a cell phone camera. So what’s really neat about the mapping and localization pipelines that we’ve been creating for this platform, and we actually have several. We can do things like improving the GPS signal, and I can go into how that works. We can also do things by basically just using videos that are shot from either GoPro cameras, or even your cell phone, like a more modern iPhone or Android platform.
And actually take that and we feed it into a neural net, which is artificial intelligence. And then we’re able to basically pull out these really cool feature sets. And we build these topometric maps. So a topometric map is basically what I like to consider a visual plaster cast of the actual environment. It’s not very, very detailed, but it’s rough enough to give you good localization information, potentially even down to the foot.
So we should be able to get you relatively good and strong accuracy, reliably where we can then position you based on some type of a camera image and knowing where that camera is and the orientation of that camera based on the image feed itself. The way we do that is by creating these really cool images and then putting them on to some type of floor plan. And so we create these kind of topometric maps, if you will. And those can be downloaded ahead of time depending on your trip, depending on where you need to go. And that can be the basis of support for your next steps in those environments.
I just want to be clear I heard what you said.
Sure.
All right. Because I’ve been looking at all of these different options out there for like way finder, internal GPS systems, whatever. I don’t know what the right terminology is. But something to help you find your way through.
A GPS.
No, GPS it’s not where I’m going. Internal buildings because, JR. and I were at the UN. And I was introducing the people at the UN to different technologies out there that were rooted in beacons having to be placed internally within a building. And then there was another one called click and go years ago, that required extensive mapping. And the mapping was so extensive that it had to be done by mobility instructors, making it a very, very heavy lift.
If I’m hearing you correctly, we could then use Google Maps and download maps from any GPS system and/or floor plans for buildings in a manner that we just upload the information into the backpack and then it would be is, I’m hearing you say, too JR. refined and updated as we collect more information from the person who’s actually walking through the environment?
Correct, yes and so what we’ve actually done is, we use 2D floor plans and we use other assets that could be provided. The model could be the individual during their first walk is actually creating the asset. But that would take some additional time. What we’ve actually been doing in our research to be totally transparent and open is we actually send out our team and basically, you can think of it as– well, you know UN, Albert. We actually did, and we’re working in cooperation with this UN right now.
We’ve actually mapped all public facing areas within the UN buildings and we achieve that within about a three hour time frame in walking through all three buildings, both inside and outside courtyards and entrances. And we were able to create one of these, what we call, nicer topometric maps based on that, which now allows us to then have a map that can basically sync with a floor plan, and create an indoor equivalent of Google Maps.
Now, what’s nice about that is we can then provide localization support in the actual UN. So as you’re walking through the UN, your camera can then be used to grab a snapshot like you were taking a picture, every so often like once a second. And based on that picture, we know exactly where you are in the UN. And we can support wherever you’d want to go, and also help you with next steps or even give you some type of a virtual tour if you want it to.
s I see the application here being very far reaching. And just to get me even more excited about the thought and concept, is this adaptable for say, non-native language speaking individuals or people with mobility issues or in instances where people just can’t get access to a help desk? Is this going to be universally applicable to people beyond those who have disabilities?
Not only do I confidently say yes, but we actually submitted this with our engineering colleague Chen Feng who I’ve worked hand in hand with this over the last two and a half years on. We submitted this approach with a global health expert at NYU named Rajesh Vaidyanathan. And we are what we think about a weeks away from getting a formal notice of award to do this exact work to support a blind visually impaired school in Thailand, so we can help visually impaired Thai students navigate their campus and actually get access to their cultural facilities, which they don’t really have access to right now.
Wow.
Really excited about that.
So JR. All of this information, I’m sure that people who are listening are very excited about learning more and staying in touch with you. If our listening audience wanted to follow you or get more information?
The best way to get a hold of me is to reach out to me through my NYU School of Medicine email address, which is johnross.rizzo@nyulangone.org. So I’ll spell that real quick, J-O-H-N-R-O-S-S-R-I-Z-Z-O @nyulangone, L-A-N-G-O-N-E. But if you just Google JR. Rizzo, NYU School of Medicine, I should pop up right away in terms of faculty and you should be able to find some of our research programs or publications.
All right JR. Thank you, brother. I really, really appreciate.
You thank you so much for having me. This was fantastic.
Thank you for your time. It was a pleasure, man.
Yeah, I appreciate all the questions. And really looking forward to hearing this and happy to come back as well. So thank you for your time and also for supporting the field.
Well, that was enlightening. I was confused about the whole thing, but you two seemed to get on really well from a nerdy scientific perspective.
Wildly intelligent.
Yeah.
I guess I wasn’t expecting him to be so damn smart. But what did you expect from a doctor?
No, he’s just remarkable. And he’s so genuine about his experiences and the sincerity with which he approaches improving lives of other people is just really right out there for us.
We have actually confirmation that they did win the award in Thailand.
Excellent.
Yeah.
Well, that is so cool.
I actually look forward to talking to JR. again.
Well, that’s today’s show. And up next is going to be Bruce Howell, who is with the Carroll Center for the Blind in Boston. My Blind Spots working closely with a new pilot program. They’ve launched on training people who happen to be blind or visually impaired on best practices for being digital accessibility testers. So we’re looking forward to that podcast hearing more about Bruce, what he’s done over the years, and what’s happening at the Carroll Center.
As we close for today, feel free to reach out to us on any of our social media platforms, LinkedIn, Twitter, and Facebook. Go to our website, myblindspot.org where you can find our podcast and our blogs, anything else that we do.
Yeah.
Our resources.
And our next blog is going to be again about situational disabilities. It’s an interesting consideration and it reinforces my position that everybody out there is temporarily able.
It does play off that.
It does. It really, really does. And it does speak to the very real importance of true inclusion of ability alongside race, gender orientation, and religion in our social and corporate cultures. Because the disability community is at the intersectionality of all other, quote-unquote, minority groups and majority groups. Anyway, that’s our show and we’re sticking to it. Please, please remember COVID is still a very real thing happening around this country and the world.
So we want to hear you, see you, and have you live a long life. So wash your hands. Really do investigations about the value of the vaccine. There are still people who are not vaccinated, who are ending up in the hospital and it is not a fun ride. Thanks again for listening, this has been the AccessAbility Works podcast.
A podcast about the possibilities of accessibility for people with disabilities. I’m Jonathan Hermits.
And I’m Albert Rizzi. Thanks for listening. Can you stay and listen to us talk stupid? Thank you for being a friend
Travel down the road and back again.
[END OF TRANSCRIPT]
Return to the AccessAbility Podcast Page