The document discusses embedding data in knitted textiles, simulating knitting digitally, reverse engineering knitting patterns from existing knits, and developing a machine learning model to recognize stitch patterns in photographs of knitting. It describes collecting a dataset of knitting images, manually labeling them with their stitch patterns, training a deep learning model on the images, and deploying the model as a web application for stitch pattern recognition. Key challenges discussed include obtaining a large enough training set and achieving high accuracy on the validation set after multiple rounds of training.

1. @web_goddess
Knitted Disruption
Kris Howard - Build Stuff 2019

3. @web_goddess
I’m a knitter

4. @web_goddess
..but I’m also a geek.

5. @web_goddess
Overview
• Embedding data in textiles
• Knitting machines and DSLs
• Simulating knitting and digital manipulation
• Reverse engineering and forensic knitting
• My KnittingML project
• Lessons from the knitting community

6. @web_goddess
Embed data

7. @web_goddess
Binary encoding
“Binary scarf” by heycarrieann

8. @web_goddess
The Viral Knitting Project

9. @web_goddess
squareup.com/market/glitchaus

10. @web_goddess
QR codes

11. @web_goddess
https://www.tempestryproject.com

12. @web_goddess
“The Sleep Blanket” by Seung Lee

13. @web_goddess
“The Neuroknitting Project” - Flickr: Mar Canet

14. @web_goddess
Knitting machines

15. @web_goddess
heartofpluto.co

16. @web_goddess
heartofpluto.co

17. @web_goddess
Kniterate

18. @web_goddess
Knitting DSLs

19. @web_goddess
Jacquard Loom

20. @web_goddess
Text patterns

21. @web_goddess
Graphs

22. @web_goddess
Stitch Maps
stitch-maps.com

23. @web_goddess
Rows 1, 5, and 9 (RS): K1, *k3, k2tog, k4, yo, p2, [k2, yo, ssk] 3 times, p2, yo, k4, ssk,
k3, repeat from * to last st, k1.
Rows 2, 6, and 10: K1, *p2, p2tog, p4, yo, p1, k2, [p2, yo, p2tog] 3 times, k2, yo, p4,
p2tog, p3, repeat from * to last st, k1.
Rows 3, 7, and 11: K1, *k1, k2tog, k4, yo, k2, p2, [k2, yo, ssk] 3 times, p2, k2, yo, k4,
ssk, k1, repeat from * to last st, k1.
Rows 4, 8, and 12: K1, *p2tog, p4, yo, p3, k2, [p2, yo, p2tog] 3 times, k2, p3, yo, p4,
p2tog, repeat from * to last st, k1.
Rows 13, 17, and 21: K1, *yo, ssk, k2, yo, ssk, p2, yo, k4, ssk, k6, k2tog, k4, yo, p2, k2,
yo, ssk, k2, repeat from * to last st, k1.
Rows 14, 18, and 22: K1, *yo, p2tog, p2, yo, p2tog, k2, p1, yo, p4, p2tog, p4, p2tog,
p4, yo, p1, k2, p2, yo, p2tog, p2, repeat from * to last st, k1.
Rows 15, 19, and 23: K1, *yo, ssk, k2, yo, ssk, p2, k2, yo, k4, ssk, k2, k2tog, k4, yo,
k2, p2, k2, yo, ssk, k2, repeat from * to last st, k1.
Rows 16, 20, and 24: K1, *yo, p2tog, p2, yo, p2tog, k2, p3, yo, p4, p2tog, p2tog, p4,

24. @web_goddess

25. @web_goddess
Industrial
Shima Seiki SDS-ONE Design Software

26. @web_goddess
DOKUknit from DOKU GmbH

27. @web_goddess
Purl
Paper by Chelsea Corvus

28. @web_goddess
http://chelsea.lol/purl-undergrad/

29. @web_goddess
KnitML
knitml.sourceforge.io

30. @web_goddess<pattern:pattern xmlns:pattern="http://www.knitml.com/schema/pattern" xmlns="http://
www.knitml.com/schema/operations" xmlns:common="http://www.knitml.com/schema/common"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://
www.knitml.com/schema/pattern http://www.knitml.com/schema/pattern-0.7.xsd http://
www.knitml.com/schema/operations http://www.knitml.com/schema/operations-0.7.xsd http://
www.knitml.com/schema/common http://www.knitml.com/schema/common-0.7.xsd" version="0.7"
xml:lang="en">
<pattern:directives>
<pattern:instruction-definitions>
<inline-instruction id="cross-2-over-2" label="2/2 LC">
<cross-stitches first="2" next="2" type="front" />
<knit>4</knit>
</inline-instruction>
<inline-instruction id="cross-2-behind-2" label="2/2 RC">
<cross-stitches first="2" next="2" type="back" />
<knit>4</knit>
</inline-instruction>
</pattern:instruction-definitions>
</pattern:directives>
<pattern:general-information>
<pattern:name>Color Sampler Swatch</pattern:name>
<pattern:description>A basic 2-color swatch sampler</pattern:description>
<pattern:dimensions />
<pattern:gauge type="stockinette">
<pattern:stitch-gauge unit="st/in">5</pattern:stitch-gauge>
<pattern:row-gauge unit="row/in">7</pattern:row-gauge>
</pattern:gauge>

31. @web_goddess
Pattern "en" {
Directives {
InstructionDefinitions {
InlineInstruction 'cross-2-over-2' [label:"2/2 LC"] { cross 2 inFrontOf 2, k4 }
InlineInstruction 'cross-2-behind-2' [label:"2/2 RC"] { cross 2 behind 2, k4 }
}
}
GeneralInformation {
Name: "Color Sampler Swatch"
Description: "A basic 2-color swatch sampler"
Dimensions
Gauge 'stockinette' {
StitchGauge: 5 stitchesPerInch
RowGauge: 7 rowsPerInch
}
Techniques {
Technique: "stranded knitting"
Technique: "cabling"
}
Author {
FirstName: "Jonathan"
LastName: "Whitall"
}
KnittingEL

32. @web_goddess
Benefits of a DSL

33. @web_goddess
Simulation
sourceforge.net/projects/knitter/

34. @web_goddess
cs.cornell.edu/projects/stitchmeshes/

35. @web_goddess
cs.cornell.edu/projects/stitchmeshes/

36. @web_goddess
cs.utah.edu/~kwu/stitchmodeling

37. @web_goddess
cs.utah.edu/~kwu/stitchmodeling

38. @web_goddess
textiles-lab.github.io/publications/2019-knitpick/

39. @web_goddess
<pattern />
???

40. @web_goddess
Could a computer
READ knitting?

41. @web_goddess

42. @web_goddess
Reverse
engineering

43. @web_goddess
Queen Susan shawl
“Queen Susan Shawl” by The Ravelry Heirloom Knitting Forum

44. @web_goddess
The Green Sweater
“The Green Sweater” by Lea Stern

45. @web_goddess
Mawson’s Balaclava
mawsons-huts.org.au

46. @web_goddess
Why not?

47. @web_goddess
My ML experience
xkcd.com/1425/

48. @web_goddess
deepknitting.csail.mit.edu

49. @web_goddess
The plan
1. Get training data
2. Label training data
3. Train ML model
4. Deploy ML model
5. Build web frontend for inference
6. Dance on the grave of 1500-year-old knitting industry 💃

50. @web_goddess
1. Get training data

51. @web_goddess
github.com/textiles-lab/knitdb

52. @web_goddess
deepknitting.csail.mit.edu

53. @web_goddess
AWS Cloud
Amazon Simple
Email Service
Email
Amazon Simple
Storage Service
AWS Lambda
Amazon Simple
Storage Service
Amazon
DynamoDB
Email Processing

54. @web_goddess

55. @web_goddess

56. @web_goddess
2. Label training data

57. @web_goddess

58. @web_goddess
Garter Stitch
All rows: Knit
Reverse Stockinette
Row 1: Purl
Row 2: Knit

59. @web_goddess
theknittingspace.com/seed-stitch-vs-moss-stitch/

60. @web_goddess

61. @web_goddess
3. Train ML model
ml.p3.2xlarge

62. @web_goddess
4. Deploy ML model

63. @web_goddess
5. Build web frontend
for inference

64. @web_goddess

65. @web_goddess

66. @web_goddess

67. @web_goddess

68. @web_goddess
Number of training images: 368
Train-accuracy=0.579545
Validation-accuracy=0.458333

69. @web_goddess
3. Train the model… again
xkcd.com/1838/

70. @web_goddess
Demo time!

71. @web_goddess
Number of training images: 900
Train-accuracy=1.000000
Validation-accuracy=0.948275

72. @web_goddess
Image providers
Alexis Nast, Alison Reeve, Amanda Blicharz, Catherine Eibner, Catherine Manning, Cheryl Dubien, Christine Jones, Clarissa Meyer,
Claudine Ong, Cornelius McCallum, Debbie Teakle, Donna Edgar, Emma Coleman, Emma McDonnell, Erin Zimmer, Fabi Meyas,
Good Wool NZ, Jana Wiese, Jane Downey, Jess Jourdan, Jessica Marshallsay, Jessica McAlinden, Judith Simpson, Karin Slegers,
Kate Peters, Katie Vhay, Laura Hampton, Lauren Burka, Lele Schirmeister, Linda van der Pal, Lisa Eckstein, Lynn Tostevin, Marg
Willis, Maria Shanley, Megan Cole, Melanie Duff, Monica PDX, Nancy Hunt, Nicole O’Hara, Paula McKinney, Rian Anderson, Rachel
Hands, Sabine Feldner, Sally Webersinke, Sian Robyns, Simeon Johnson, Sophie Kunze, Tammy Meyer, Tammy Vipond, Tehmina
Gladman, Theresa Norrmen, Theresa Sovacool, Ulrika Malmgren
Labellers
Amelia Hodsdon, Amy Allenspach, Carina Zona, Celia King, Donna Edgar, Jen Luker, Jody Podbury, Kristen Symonds, Lele
Schirmeister, Lesley Diskin, Liz Evatt, Marg Willis, Merry Glennie, Tamsin
Guidance
Aparna Elangovan, Gabe Hollombe

73. @web_goddess
Lessons

74. Humility
Flickr: bekathwia

75. Documentation
Flickr: malglam

76. Invest in your tools
Flickr: markseton

77. Welcome newbies

78. @web_goddess
Thank you!