Más contenido relacionado

Más de John Breslin(20)


Old Ireland in Colour

  1. Old Ireland in Colour (OIIC) Professor John G. Breslin VistaMilk and Insight SFI Research Centres Data Science Institute NUI Galway
  2. For most DSI peeps, you have probably never heard of her She was an Irish author and storyteller born in Kerry Most non-millennial Irish people studied her in school (the Irish Times says that “her mournful memoir was required reading for a generation of Irish school students”) Previously, all we had was that sad book and some black and white photos of her This is Peig Sayers
  3. This is DeOldify A deep learning model to colourise (old) black and white photos Created by Jason Antic @citnaj
  4. This is Peig Sayers with DeOldify [plus TorchWarp (DL, morphs two images)]
  5. And it’s not just photos… You can DeOldify video too!
  6. Who made DeOldify? Made by Jason Antic, deep learning researcher/programmer Took on the idea of using DL for image colourisation as a hobby/project when he was undertaking the course in Spring/Summer 2018 Met up with Dana Kelley (collaborator on the project, left) and Jason Antic when I was in San Diego in June 2019
  7. ● GAN requires setting up the training of a colourising model that involves a second model —  the “critic”  —  that basically is there to “criticize” the colourisations and teach the “generator” to produce better images ● Since the “critic” model is also a neural network, it can pick up on a lot of the nuances regarding what makes something look more “realistic” (that simpler methods just cannot do) ● Output is often more colourful and convincing compared to previous efforts (i.e. deep learning models for colourisation) How does it work? (GAN — Generative Adversarial Network) [more here]
  8. More on how I got into this “Donegal family research leads to amazing colour update for black & white photos” ily-research-leads-to-amazing-colour-update-for-black-white-p hotos.html
  9. Some favourites 1910 1911
  10. 1902 1896 1914
  11. 1963 Galway ~1942 UPenn
  12. 1906 1882
  13. The Stoneys, Aula Maxima, Alice Perry Edith Stoney = first female medical physicist in the world Florence Stoney = first female radiologist in UK or Ireland George Johnstone Stoney = originator of the term “electron”, NUI Galway professor Alice Perry = first female engineering graduate in UK or Ireland
  14. 1902
  15. 1932 (two Galway men)
  16. Often requires colouring in of arms, legs or faces: sometimes a lot of them!
  17. Before you ask, historical accuracy is a challenge (“Golden” Gate Bridge) But more about how to tackle that later...
  18. Mass colourisation using DeOldify ● Running on my MSI Aegis Ti3 8th desktop ● Using a local installation of DeOldify’s Docker/API instance on Ubuntu with NVIDIA Docker and CUDA (there’s also a Jupyter interface) ● Used FlickrDump to get images from NLI, PRONI and wget from Dúchas ● Processes a photo every second or so ● 20,000 done in a number of hours ● Downloaded larger image banks for future retraining
  19. 2729 photos
  20. 2778 photos
  21. 3267 photos
  22. 11254 photos
  23. Social media: >15k followers in three months (Facebook, Twitter, Instagram)
  24. 2.2M engagements on Twitter in 3 months
  25. Facebook has the most followers (nearly 8k likes)
  26. Instagram is quieter but still busy
  27. Inspiration for new “countries in colour” accounts from various people around the world Pakistan, New Zealand, Argentina...
  28. Back to the historical accuracy ● Some knowledge of the clothes someone/some group wore, paintings, traditions, etc. (e.g. red Galway shawls) ● Usually requires some additional touchup or manual adjustment ● New techniques allow you to spot touch areas with a dot of colour and then spread/apply that to a region, or even to the same region across video frames ● But the most standard way is to fire up PhotoShop
  29. Countess Markievicz
  30. Amelia Earhart
  31. Titanic
  32. DeOldify links DYK DeOldify only kicked off as a project in the summer of 2018! (code, etc. from Jason Antic) (DeOldify images on Google Colab) (DeOldify videos on Google Colab) (commercial app, uses DeOldify, third party) (by John)
  33. DeepRemaster: Temporal Source-Reference Attention Networks for Comprehensive Video Enhancement Interactive Deep Colorization See also
  34. Thank you Any questions? (Irish photos from Dúchas used under a CC BY-NC 4.0 license, most other Irish photos used are public domain)