Artificial Vision & Neuroprosthetics
•
3 gefällt mir•1,736 views
Current Status on Artificial Vision & Neuroprosthetics
Empfohlen
Weitere ähnliche Inhalte
Was ist angesagt?
Was ist angesagt? (20)
Andere mochten auch
Andere mochten auch (20)
Ähnlich wie Artificial Vision & Neuroprosthetics
Ähnlich wie Artificial Vision & Neuroprosthetics (20)
Mehr von Jeanna Leaves
Mehr von Jeanna Leaves (13)
Kürzlich hochgeladen
Kürzlich hochgeladen (20)
Artificial Vision & Neuroprosthetics
- 1. J e a n n a N i k o l o v - R a m i r e z . N e u r o s c i e n c e 2 0 1 4 S u p e r v i s o r : M . E r n s t . 2 3 . D e c . 2 0 1 4 Artificial Vision & Neuroprosthetics
- 2. Motivational statement § Interest in aesthe+cs and visual percep+on § Contribu+on of aesthe+cs and form to insight § Interest in robo+cs and informa+on processing § Sensory subs+tu+on § Inves+ga+ng advancements and challenges in the field of human-‐ brain interfaces § More specifically nascent field of Visual Neuroprosthe+cs 30/12/14 2 Aesthe+cs and Visual Percep+on Neuroscience and Informa+on processing Robo+cs and AI NeuroScience, MEi:CogSci Nikolov 2014.12.23
- 3. Outline § History of Ar+ficial Vision § Visual Apparatus and Re+na § Current Approaches in Prosthe+c Rehabilita+on § Epire+nal implants § Subre+nal implants § Transchoroidal prostheses § Op+c nerve prostheses § Cor+cal and LGN implants § Advantages and Drawbacks Comparison § Conclusions and Further Work § References § Extra: Bach-‐y-‐Rita and Neuroplas+city 30/12/14 3 NeuroScience, MEi:CogSci Nikolov 2014.12.23
- 4. connection with neuroscience 30/12/14 NeuroScience, MEi:CogSci Nikolov 2014.12.23 4 Neural Processing of Visual Informa+on Dana Founda+on Report Neuroscien+fic Advances Interfacing
- 5. History of artificial Vision § 1755 Charles LeRoy, first +me electrical device restored a flicker of visual percep+on § 1929 Foerster: Electrical s+mula+on of the visual cortex (occipital lobe) resulted in a blind pa+ent seeing a spot of light (phosphene). § 1960ies Giles Brindley’s implanta+on of an 80-‐ electrode device onto the visual cortex of a blind pa+ent renewed the possibili+es of ar+ficial vision restora+on. § 1970ies Dobelle brain implants § 2013 FDA approval of first Re+nal Prosthesis System. Lorach, H., et al. Neural stimulation for visual rehabilitation: Advances and challenges. J. Physiol. (2012), http:// dx.doi.org/10.1016/j.jphysparis.2012.10.003
- 6. Human Visual Apparatus § Photoreceptor § Rods (low light) and cones (color) § Bipolar cells § Ganglion cells § Axons form the op+c nerve to lateral geniculate nucleus of thalamus 30/12/14 NeuroScience, MEi:CogSci Nikolov 2014.12.23 6
- 7. Approaches in Prosthetic rehabilitation § Epire+nal implants § Subre+nal implants § Transchoroidal prostheses § Op+c nerve prostheses § Cor+cal and LGN implants 30/12/14 NeuroScience, MEi:CogSci Nikolov 2014.12.23 7
- 9. Electrodes resolution 2010, hap://images.dailytech.com/nimage/14077_large_vision_resolu+on.png 30/12/14 NeuroScience, MEi:CogSci Nikolov 2014.12.23 9
- 10. ARGUS I ARGUS II IRIS, Intelligent Medical Implant EPIRET-3 STS OPTIC NERVE IMPLANT DOBELLE Cortical implant BOSTON RETINA IMPLANT Subret. RETINA IMPLANT AG STANFORD SUBRET. MIcrophoto diode ASR Vision Institute Paris UTAH ELECTRODE ARRAY
- 11. strategies advantages and drawbacks • Re+nal and op+c nerve implanta+ons are safer than brain s+mula+on approaches. • Implant stability has been demonstrated in all techniques. • The electrode-‐+ssue contact is improved in subre+nal approaches. • The processing complexity increases in higher visual streams. • The poten+al acuity restora+on is highly dependent on the ability to s+mulate a limited corresponding visual field. • Re+nal and op+c nerve strategies are only suited for pa+ents with intact ganglion cells and op+c nerve (re+ni+s pigmentosa and AMD). • Brain s+mula+on in contrast can be used in any visual impairment. 30/12/14 NeuroScience, MEi:CogSci Nikolov 2014.12.23 11
- 12. Conclusions and Further Questions § Research is nascent. § Challenges include: § Accoun+ng for eye movements, (use eye tracking system to select in real +me the part of the image corresponding to gaze direc+on) § research into signal processing from photoreceptors, § target specific cell types independently (e.g. ON and OFF), § electrode miniaturiza+on, § material op+miza+on, § mul+plexing of s+mula+on channels , § developing of real +me processing algorithms (adequate filtering) to provide relevant physiological s+muli, encoding of visual informa+on into electrical s+muli. § Other promising strategies are emerging: § Optogene+cs § Cell therapy 30/12/14 MEi:CogSci MoPE Nikolov 2014.12.17 12
- 13. References 1. Lorach, H., Marre, O., Sahel, J. A., Benosman, R., & Picaud, S. (2013). Neural s+mula+on for visual rehabilita+on: Advances and challenges. Journal of Physiology-‐Paris, 107(5), 421-‐431. Chicago 2. Kien, T. T., Maul, T., & Bargiela, A. (2012). A review of re+nal prosthesis approaches. In Interna+onal Journal of Modern Physics: Conference Series (Vol. 9, pp. 209-‐231). World Scien+fic Publishing Company. 3. Weiland, J. D., Liu, W., & Humayun, M. S. (2005). Re+nal prosthesis. Annu. Rev. Biomed. Eng., 7, 361-‐401. 4. hap://www.the-‐scien+st.com/?ar+cles.view/ar+cleNo/41324/+tle/Neuroprosthe+cs/ 5. hap://www.the-‐scien+st.com/?ar+cles.view/ar+cleNo/41052/+tle/The-‐Bionic-‐Eye/ 6. hap://isites.harvard.edu/fs/docs/icb.topic793620.files/Re+nal_ar+ficial.pdf 7. hap://archive.wired.com/wired/archive/10.09/vision_pr.html 8. hap://www.bostonre+nalimplant.org/assets/Uploads/KellyTBME2011.pdf 9. hap://biomed.brown.edu/Courses/BI108/BI108_1999_Groups/Vision_Team/Cor+cal.htm 10. hap://www.lems.brown.edu/~jgr/cor+cal_prosthesis_proposal.htm 11. hap://www.technologyreview.com/news/407739/brain-‐implants-‐to-‐restore-‐vision/ 12. hap://www.dana.org/Publica+ons/ReportOnProgress/ Ar+ficial_Sight_Restora+on_of_Sight_through_Use_of_Argus/ 13. hap://www.natureasia.com/en/research/highlight/8524 30/12/14 MEi:CogSci MoPE Nikolov 2014.12.17 13
- 14. MEi:CogSci MoPE Nikolov 2014.12.17 30/12/1414 THANK YOU!
- 15. Epiretinal: Argus I and Argus II Humayun et al. hap://www.expertsmind.com/topic/neuroscience/re+nal-‐processing-‐93034.aspx Pros: • S+mula+ng close to photoreceptors takes advantage of na+ve processing power in the thalamus and cortex • Surgical complica+ons not necessarily as significant as cor+cal approach Cons: • Requires func+onal op+c nerve pathway • May s+mulate op+c nerve fibers rather than cell bodies • Difficult to adhere electrode array to re+na First in 2002 30/12/14 NeuroScience, MEi:CogSci Nikolov 2014.12.23 15
- 16. SUBRETINAL hap://www.bostonre+nalimplant.org/assets/Uploads/ KellyTBME2011.pdf Pros: • Inserted below re+na • Maintained between the choroid and the re+na itself • No addi+onal tack for fixa+on • Posi+on increases implant stability but risk of re+nal detachments Cons: • Subre+nal s+mula+on threshold were found to be lower than for epire+nal s+mula+on hap://optobionics.com/ asrdevice.shtml 30/12/14 NeuroScience, MEi:CogSci Nikolov 2014.12.23 16
- 17. Transchoroidal prostheses Bionic Vision Australia Media Release, haps://app.box.com/s/bq9jt8g1uvs014dex84s/1/2495873247/21429693717/1 Pros: • S+mulate re+na from the outer part • Easier implanta+on, low +ssue damage • No risk of re+nal detachment Cons: • Requires higher current intensi+es to elicit visual percepts because of the increased distance between electrodes and inner re+nal neurons 30/12/14 NeuroScience, MEi:CogSci Nikolov 2014.12.23 17
- 18. OPTIC Nerve Prostheses C-‐Sight project hap:// contest.techbriefs.com /2012/entries/medical/ 2933 Pros: • Surgical complica+ons not necessarily as significant as cor+cal approach Cons: • Requires func+onal op+c nerve pathway • Will require complex electrode array to provide any useful paaerned vision • Very difficult surgical access 30/12/14 NeuroScience, MEi:CogSci Nikolov 2014.12.23 18
- 19. Cortical and LGN IMplants Pros: • Only approach for individual with non-‐func+onal re+nas and/or op+c nerves • Implant site robust and protected by skull • Phosphene thresholds are low (1 -‐ 10 uA range) • Ac+vates electrodes on surface of visual cortex Cons: • S+mula+on site far from photoreceptors (no re+nal or thalamic processing), thus some visual processing is missing • Problems of mul+ple feature representa+ons in V1 (color, lines, mo+on, ocular dominance) • Requires permanent skull interface • Highly invasive with major risks of infec+on and inflamma+on • Cellular death around the electrodes occurring amer electrical s+mula+on hap://biomed.brown.edu/Courses/ BI108/2006-‐108websites/group03re+nalimplants/ dobell.htm 30/12/14 NeuroScience, MEi:CogSci Nikolov 2014.12.23 19
- 20. Bach-y-Rita and Neuroplasticity • haps://www.youtube.com/ watch?v=7s1VAVcM8s8
- 21. ARGUS II
- 23. EYE and Retina
Hinweis der Redaktion
- Interest in aesthetics and visual perception Contribution of aesthetics and form to insight Interest in robotics and information processing Investigating advancements and challenges in the field of human-brain interfaces More specifically visual Neuroprosthetics
- In 1755, French physician and scientist Charles Leroy discharged the static electricity from a Leyden jar—a precursor of modern-day capacitors—into a blind patient’s body using two wires, one tightened around the head just above the eyes and the other around the leg. The patient, who had been blind for three months as a result of a high fever, described the experience like a flame passing downwards in front of his eyes. This was the first time an electrical device—serving as a rudimentary prosthesis—successfully restored even a flicker of visual perception. The history of the cortical prosthesis begins in 1929 when Foerster investigated the effects of electrical stimulation of the occipital lobe of the human cortex [3]. He found that this stimulation caused a subject to "see" a small point of light, later called a "phosphene". This result was reproduced many times after the original experiment with both sighted subjects and blind subjects. The idea that concurrent stimulation of many sites in the brain could produce a single coherent image was postulated as early as 1953 by Krieg [4]. Because there is rough retinotopy in the visual cortex, Krieg thought it would be possible to use this technique to restore sight to the blind. Giles Brindley in the 1960s, and William Dobelle (1941-2004) et al. in the early 1970s
- Blindness affects tens of million people worldwide and its prevalence constantly increases along with population aging. Cataract 51 % cases, The remaining causes of acquired blindness are glaucoma, age-related macular degeneration (AMD), diabetic retinopathy, and retinitis pigmentosa (RP) 125 Mio photoreceptors Degenerative diseases primarily affect the photoreceptors, ultimately resulting in significant loss of vision. In some pathologies leading to vision loss, prosthetic approaches are currently the only hope for the patient to recover some visual perception.
- epiretinal Implants (on the retina), subretinal Implants (behind the retina), and suprachoroidal implants (above the vascular choroid) Stimulating the optic nerve directly is also possible, although the high density of nerve fibers is an issue for stimulation control. And finally, it is possible to stimulate brain structures such as the lateral geniculate nucleus (LGN) or the visual cortex directly, even in case of complete retinal degeneration or optic nerve injury. However, these strategies are much more invasive. In all cases, the device consists in a photosensitive part – i.e. camera – a processing stage and an array of electrodes in contact with the targeted structure In the experimental cortical implant approach one uses an array of electrodes placed in direct contact with the visual cortex. This is also called a neuroprosthesis or brain implant. Pioneered by Giles Brindley in the 1960s, and William Dobelle (1941-2004) et al. in the early 1970s. Research into cortical implants is also done by Richard Normann, Mohamad Sawan and others. The cortical implant is an invasive approach, requiring major surgery. Maximum resolution of the implants is so far on the order of 10 × 10 pixels. Individual pixels are perceived as phosphenes (sensations of light, flashes). See also the brain implant page for more information and discussion. A related research topic is the development of the retinal implant for artificial sight. The retinal implant approach may in the future help in treating blindness resulting from malfunctioning of the retina. It will not help with blindness resulting from optic nerve damage (e.g., due to diabetes or glaucoma) or brain damage. First, the retina can be stimulated in case of ganglion cell survival and preservation of the information flow through the optic nerve. Stimulating the optic nerve directly is also possible, although the high density of nerve fibers is an issue for stimulation control. And finally, it is possible to stimulate brain structures such as the lateral geniculate nucleus (LGN) or the visual cortex directly, even in case of complete retinal degeneration or optic nerve injury. However, these strategies are much more invasive. In all cases, the device consists in a photosensitive part – i.e. camera – a processing stage and an array of electrodes in contact with the targeted structure
- - Sclera: white of the eye (from Greek sclerus, hard) Choroid: vascular layer of the eye, containing connective tissue, and lying between the retina and the sclera. Retina: light-sensitive layer of tissue The lateral geniculate nucleus (LGN) (also called the lateral geniculate body or lateral geniculate complex) is a relay center in the thalamus for the visual pathway. It receives a major sensory input from the retina. The LGN is the main central connection for the optic nerve to the occipital lobe. Each LGN has six layers of neurons (grey matter) alternating with optic fibers (white matter). Here, we review the latest advances in visual prosthetic strategies with their respective strength and weakness. electrically stimulate neurons along the visual pathway Ocular approaches(less invasive) target the remaining retinal cells whereas brain stimulation aims at stimulating higher visual structures directly. (glaucoma, whereconnection between retina and brain is lost)
- Australia 36 x 36 supposedly in 2013 Bionic Vision Australia (BVA) and its academic partner, the University of New South Wales. The pair unveiled their "first advanced prototype", the culmination of efforts financed by a $42M USD research grant from the Australian government. - See more at: http://www.dailytech.com/Bionic+Vision+Unveils+Advanced+Prototype+Electronic+Eyeball/article18021.htm#sthash.oahAkSQc.dpuf
- Fig. 5. Summary picture of the different visual prosthetic devices discussed here. Argus I device from second sight (from Humayun et al. (2003)). The Argus II devicewith 60 electrodes, first prototype on the market (from Humayun et al. (2012)). The IRIS device from Intelligent Medical Implant AG (from Hornig et al. (2008)). The EPIRET-3 device (from Roessler et al. (2009)). The STS suprachoroidal implant (from Fujikado et al. (2011)). The optic nerve implant developed by Louvain’s university(from Veraart et al. (2003)). (G) Dobelle’s seminal cortical implant (from Dobelle (2000)). (H) The Boston Retina Implant Project device (from Rizzo (2011)). The subretinal device from Retina Implant AG (from Zrenner et al. (2010)). (J) The subretinal microphotodiode array developed by Pr. Palanker’s group at Stanford university (from Wang et al. (2012). (K) The ASR device from Optobionics (from Chow et al. (2004)). (L) The diamond coated subretinal implant from the Vision Institute of Paris. (M) The Utah electrode array for cortical stimulation (from Normann et al. (2009)). All figures were reproduced with permission from their respective editors.
- Prosthetic strategies advantages and drawbacks. Retinal and optic nerve implantations are safer than brain stimulation approaches. Implant stability has been demonstrated in all techniques however, the electrode-tissue contact is improved in subretinal approaches. The processing complexity increases in higher visual streams so that retinal approaches only need limited computation. The potential acuity restoration is highly dependent on the ability to stimulate a limited corresponding visual field. Retinotopic area is higher in the brain and smaller in the optic nerve, therefore resulting in different angular resolution for a given electrode size. Finally, retinal and optic nerve strategies are only suited for patients with intact ganglion cells and optic nerve – mainly retinitis pigmentosa and AMD. Brain stimulation in contrast can be used in any visual impairment when it is the only solution.
- Optogenetics (from Greek optos, meaning "visible") uses light to control neurons which have been genetically sensitised to light.
- Epiretinal implants electrically target the ganglion cell layer. A matrix of electrodes is directly fixed on the surface of the retina with a tack and connected to a stimulator receiving data and power through coil–coil interaction and radio-frequency (RF) signal. Humayun et al. were the pioneer of epiretinal implants (Humayun et al., 2003, 2009, 2012). The first epiretinal device tobe chronically implanted in patients – the Argus I – developed by Second Sight Medical Products was composed of 16 electrodes (Humayun et al., 2003; Caspi et al., 2009). Their report confirmed that light perception could be achieved through epiretinal stimulation. The implanted patient was able to recognize shapes, gratings orientations, and had a restored visual acuity of 20/3240. The next generation of their epiretinal device named Argus II was designed to reach a higher resolution. Wireless data transfer Only 16 electrodes Soon 1024 (32x32) complex visually guided tasks such as object localization (96% of subjects), motion discrimination (57%), and discrimination of oriented gratings (23%) (Humayun et al., 2012). Fig. 4 describes the Argus II device containing a 6 10 electrode matrix implanted in 30 subjects from 2007 to 2009. On February 14, 2013, the US Food and Drug Administration (FDA) approved the Argus II Retinal Prosthesis System. Recently received CE mark for commercialization in Europe and will be sold around 100,000$. Advantages: direct immersion in the vitreous dissipates heat from electrical stimulation. easily implantable and is less likely to induce retinal detachment or injury compared to subretinal prostheses Disadvantages: do not benefit the inner layers of the retina that naturally act as an amplification and encoding Medical Implants (IMIs) in Switzerland developed a 49 platinum electrode prosthesis in which power is transmitted through a RF-link and data, through infra-red pulses. Four patients were chronically implanted with this device (Hornig et al., 2008) and were able to perform localization tasks and recognize simple light Patterns EPI-RET group in Germany also performed clinical trials with their EPIRET3 device implanted in six patients (Klauke et al., 2011).
- In 2001, Optobionics, Inc. developed the first implanted subretinal device called Artificial Silicon Retina (ASR). This device consisted in a 2 mm diameter autonomous array of 5000 photodiodes directly converting light into electrical stimulation. This very elegant strategy did not require any power supply nor data transmission to the chip. Once implanted, the device was completely autonomous, thereby limiting the risks of complications. Six patients implanted The Boston Retinal Implant Project that started in the 80s intends to achieve maximum development before starting clinical trials in human (Rizzo, 2011). They developed a first generation of implant containing 15 electrodes that were implanted in animals for biocompatibility and insulation assessment. Their next generation will contain more than 200 electrodes to provide useful perceptions to human patients. Hermetic retinal prosthesis and associated primary power and data coils. The implant on the left is a prototype of the device in Fig. 4, shown attached to a plastic model eye. The gold power and data secondary coils are formed on a sphere to match the eye’s curvature. The titanium case with welded lid, hermetic feedthrough, and epoxy header protects the internal circuitry. The electrode array is out of view over the top of the model eye. The primary coils on the right are potted in PDMS.
- Suprachoroidal (above the vascular choroid) Osaka, Korea, Australia BVA Bionic Vision Australia is also developing suprachoroidal devices. In initial studies in cats, they showed that they could evoke cortical activity by stimulating the retina from outside the sclera
- Optic nerve conveys the information of the entire visual field in a very small area. It is possible to stimulate peripheral and central vision at the same time. However, this nerve fiber concentration is also a disadvantage for very focal stimulation as more than 1 million axons are contained into the 2 mm diameter optic nerve. Japan an Chinese initiative – the C-sight project – is also developing optic nerve stimulation (Chai et al., 2008; Wu et al., 2010). Instead of surface stimulation the authors designed penetrating electrodes developed image processing strategies in order to encode complex visual scenes with a limited number of pixels. but that stimulation design will remain a major challenge to achieve fine spatial resolution rehabilitation in patients.
- Whenever retinal ganglion cells degenerate or after optic nerve injury, it is no longer possible to use the previous strategies. This is the case for glaucoma and optic neuropathy. Brain stimulation becomes the only available strategy for prosthetic visual rehabilitation. brain implant or cortical implant provides visual input from a camera directly to the brain via electrodes in contact with the visual cortex at the backside of the head. A computer is used to process the sensory streams, as is typical for a brain-computer interface (BCI). The seminal work of Brindley and Lewin (1968) followed by Dobelle et al. (1974); Dobelle (2000) were the first attempts in providing a functional cortical prosthesis. Dobelle’s implant was placed on the surface of the visual cortex in eight blind patients. Some of them were implanted for 20 years without infection or other complication. With this device containing 64 electrodes, one patient was able to reach 20/1200 visual acuity. With a digital zooming function, he was even able to recognize 2-inch high letters at 5-feet distance corresponding to 20/400 visual acuity. This patient had been able to learn to interpret this stimulation in one day and could to use it for 20 years. He was able to recognize characters and navigate in a room, performing complex tasks such as finding a hat and placing it over a mannequin’s head. Utah Electrode Array consists in a device with 100 electrodes at the tip of acute pillars (LGN) is also under investigation. It presents the advantage of targeting relatively simple and well characterized cells compared to cortical neurons.
- 1969, Paul Bach-y-Rita He is seen as the first to propose the concept of sensory substitution to treat patients with disabilities, often those caused by neurological problems. One of the first applications of sensory substitution he created was a chair which allowed blind people to 'see'. The trials he conducted in 1969 are now regarded to be the first form of experimental evidence for neuroplasticity and the feasibility of sensory substitution.[6] Later in his career, Bach-y-Rita created a device which enabled patients with damaged vestibular nuclei to regain their ability to remain balanced, by using an electrical stimulator placed on the tongue which reacted to a motion sensor affixed to the patient. This application enabled patients to remain balanced without the equipment after several weeks use.[7]