Jedi Mind Tricks in Git

Jedi Mind Tricks in Git
Help me Obi-Git, you’re my only hope

Who are we?
@jankrag - @randomsort
Git Trainers

“Oh my. Software development sound rather
perilous. I can assure you they will never get me
on one of those dreadful Computers”
- C3PO

this -is a command
This is the console output of a command
It can be coloured as well
this completed successfully.
#!/foo/bar
This is the content of a script
[config]
Or the content of a config file
hello=git merge
Conventions

What are git hooks?
We can jam a hook into Git’s normal flow

Git hook control flow
CompleteFinalizeWrite MsgCommitStage
precommit
Prepare
commit
msg
Commit
msg postcommit
Notification
Actionable

Yes - but how do I get started?
git init

Making Git hooks
ls .git/hooks
applypatch-msg.sample commit-msg.sample post-update.sample
pre-commit.sample prepare-commit-msg.sample
pre-rebase.sample pre-applypatch.sample pre-push.sample
update.sample

Client side vs server side hooks
precommit
prepare-commit-msg
commit-msg
post-commit
post-checkout
pre-rebase
pre-receive
update
post-receive

The truth is out there...
Specifically on master

pre-commit hook
● First thing that happens after git commit
● Able to abandon the commit
● We have not got any arguments

Where are we at?
git symbolic-ref --short HEAD
master

pre-commit hook
#!/bin/bash
# Check where our HEAD is at
if [ "$(git symbolic-ref --short HEAD)" == "master" ]; then
echo "This is not the branch you are looking for"
exit 1
fi
exit 0

cp pre-commit .git/hooks/pre-commit
git checkout master
git commit -am “I want to commit this”
This is not the branch you’re looking for

Well, I should be able to commit on master
git checkout master
git commit -n -am “I want to commit this”
[master 5d5308b] I want to commit this
1 file changed, 0 insertions(+), 0 deletions(-)
create mode 100644 foo.md

The mission
● I want to reference an issue in my commit messages
● If I do not reference an issue in my repository - abandon my commit

Where are our issues?
Git remote show origin
* remote origin
Fetch URL: git@github.com:RandomSort/hooks
Push URL: git@github.com:RandomSort/hooks
HEAD branch: master
Remote branch:
master tracked
Local branch configured for 'git pull':
master merges with remote master
Local ref configured for 'git push':
master pushes to master (up to date)

I <3 APIS
curl -s https://api.github.com/repos/$repo/issues/$issue_number
{
"url":
"https://api.github.com/repos/RandomSort/hooks/issues/1",
"repository_url":
"https://api.github.com/repos/RandomSort/hooks",
"labels_url":
"https://api.github.com/repos/RandomSort/hooks/issues/1/labels
{/name}",
....
TONS OF JSON :D

commit-msg hook
#!/bin/bash
# $1 is the temp file containing the commit message
issue_number=`cat $1 | xargs | sed 's/.*#([0-9]*).*/1/'`
repo="`git remote show origin | grep "Push" | xargs | sed
's/.*:(.*)/1/'`"
curl -s https://api.github.com/repos/$repo/issues/$issue_number |
grep -q ""Not Found"" > /dev/null
exit $((1 -$?))

So now we’ve done something sane ..
● Git hooks can do useful stuff
● Everything in moderation
● (bust the myth, then go all in)

Trust your workflow
"Fear is the path to the dark side"

Branch based delivery
C1 C2 C3 C4
ready/feature
master
HEAD
git push origin feature:ready/feature

#serverless
These are not the servers you are looking for

The mission
● I want to work on a feature branch never on master
● I want to commit work referencing issues
● I want to be able to run my Continuous Integration setup
● I deliver through ready branches
● Only allow integration if
○ Tests succeed
○ it’s a fast forward merge

Where are we at (again)?
git symbolic-ref --short HEAD
ready/foo
git rev-parse --show-toplevel
/home/randomsort/repos/hooks
.. And where’s our stuff?

We only want to do fast forward merges
● It’s prettiest
● We prefer to have our automation engine only do trivial merges
C1 C2 C3 C4
Feature#79
master
HEAD

How fast can we go?
git rev-list --maxcount 1 master..ready/foo
C1 C2 C3 C4
ready/foo
master
HEAD

Putting it together
#!bin/bash
if [ not on ready branch ]
do nothing
fi
if [ not fast forwardable ]
cleanup
fi
run tests
if [ successful ]
merge
cleanup
else
cleanup

Cleaning up fails
#We can't do a fast forward merge so let's abort this and move some
stuff around
failbranch=`echo $branch | xargs | sed 's/ready/failed/'`
echo "Unable to fast forward master to $branch leaving state on
$failbranch"
git checkout -b $failbranch
git branch -D $branch

Running
git checkout -b ready/feature
Switched to a new branch 'ready/feature'
Running tests
Testing has failed, leaving state on failed/feature
Switched to a new branch 'failed/feature'
Deleted branch ready/feature (was 462b135).

Running with successful tests
git checkout -b ready/feature
Switched to a new branch 'ready/feature1'
Running tests
Switched to branch 'master'
Updating e28500e..06a16b1
Fast-forward
temp.md | 0
1 file changed, 0 insertions(+), 0 deletions(-)
create mode 100644 temp.md
Deleted branch ready/feature1 (was 06a16b1).

It works on my machine
"I find your lack of faith disturbing"

TATOOINE - LARS OWENS HOMESTEAD
OWEN:
I have no need for a protocol droid.
C3P0:
Sir -- not in an environment such as this -- that's why I've also been
programmed for over thirty secondary functions that...
OWEN:
What I really need is a droid that understands the binary language of
moisture vaporators.

... secondary functions
"I have no need for a protocol droid."

Binary is annoying...
git diff wordfile.docx
diff --git a/wordfile.docx b/wordfile.docx
index d24c436..3c4aa59 100644
Binary files a/wordfile.docx and b/wordfile.docx differ
No idea what changed...

Teaching git new tricks
Combining two features:
● Git attributes
○ Assign behaviour per file or path
● Custom drivers
○ Define new behaviour

Git attributes
Each line in an attributes definition is of form:
pattern attr1 attr2 ...
Where a attribute can be set, unset or assigned a value:
*.txt text
*.jpg -text
*.sh text eol=lf

Git attributes - global
Default location:
$XDG_CONFIG_HOME/git/attributes.
Fallback:
$HOME/.config/git/attributes
If you want to set the path you can use:
git config --global core.attributesfile <path>
git config --global core.attributesfile ~/.gitattributes
e.g.

Git attributes - local
Just add definitions to a local .gitattributes file.
Can be set in root folder and in subfolders

Git attributes
commonly used for changing (force'ing):
● CRLF behaviour
● text / binary behaviour
But that is not Jedi enough ...... so we will skip those

Diff driver
Set as a config parameter like diff.foo
[diff "hexify"]
binary = true
textconv = hexdump -v -C

Using the new driver is easy
In .gitattributes:
*.bin diff=hexify
Note - we need both:
● Driver definition in config
● Path attribute that tells diff to use it

But hexdump is still pretty useless
00000000 89 50 4e 47 0d 0a 1a 0a 00 00 00 0d 49 48 44 52 |.PNG........IHDR|
00000010 00 00 03 20 00 00 02 80 08 02 00 00 00 eb 79 8b |... ..........y.|
00000020 65 00 00 00 09 70 48 59 73 00 00 0b 13 00 00 0b |e....pHYs.......|
00000030 13 01 00 9a 9c 18 00 00 00 06 62 4b 47 44 00 ff |..........bKGD..|
00000040 00 ff 00 ff a0 bd a7 93 00 00 8a b5 49 44 41 54 |............IDAT|
00000050 78 da ec bd 09 5b 53 57 f7 fe df f7 55 32 9d cc |x....[SW....U2..|
00000060 33 09 99 e7 79 20 09 19 50 91 41 90 41 26 41 44 |3...y ..P.A.A&AD|
00000070 44 14 11 41 04 04 ad b5 b5 b6 b5 b5 56 ed ef 85 |D..A........V...|
00000080 fd 17 a5 7f bf 3e 4a 4e 42 38 c9 39 27 e7 be ae |.....>JNB8.9'...|
00000090 cf d5 cb c7 47 25 39 67 ef b5 ee bd f7 da f7 fa |....G%9g........|
000000a0 ae a7 e7 7b 00 00 00 00 00 c0 21 df e1 11 00 00 |...{......!.....|
000000b0 00 00 00 40 60 01 00 00 00 00 40 60 01 00 00 00 |...@`.....@`....|
000000c0 00 40 60 01 00 00 00 00 00 08 2c 00 00 00 00 00 |.@`.......,.....|
000000d0 08 2c 00 00 00 00 00 08 2c 00 00 00 00 00 00 81 |.,......,.......|
000000e0 05 00 00 00 00 00 81 05 00 00 00 00 00 81 05 00 |................|
000000f0 00 00 00 00 20 b0 00 00 00 00 00 20 b0 00 00 00 |.... ...... ....|
00000100 00 00 20 b0 00 00 00 00 00 20 b0 00 00 00 00 00 |.. ...... ......|
00000110 00 04 16 00 00 00 00 00 04 16 00 00 00 00 00 04 |................|
<pages and pages without end>

Recipe for success
For each binary type:
1. Find a tool that extracts useful info from a file
2. Set up a diff driver to use it
3. Profit
4. Don't give in to the dark side

My brain parses markdown quite fine...
*.docx diff=pandoc2md
[diff "pandoc2md"]
textconv=pandoc --to=markdown
brew install pandoc
For reference:
.gitattributes
config

git diff wordfile.docx
index d24c436..48249e3 100644
--- a/wordfile.docx
+++ b/wordfile.docx
@@ -1,8 +1,8 @@
-Headline
+Hi Git Merge
This is some simple text
-*and another important paragraph in italic*
+*and another* **important** *paragraph in italic*
Word to MarkDown

git diff --word-diff=color wordfile.docx
index d24c436..48249e3 100644
--- a/wordfile.docx
+++ b/wordfile.docx
@@ -1,8 +1,8 @@
HeadlineHi Git Merge
This is some simple text
*and another* **important** *paragraph in italic*
Word to MarkDown

PDF files
*.pdf diff=pdfconv
[diff "pdfconv"]
textconv=pdftohtml -stdout
brew install pdftohtml
<BODY bgcolor="#A0A0A0" vlink="blue" link="blue">
<A name=1></a>Headline 
This is some simple text 
-and another paragraph 
-and stuff 
+and another important paragraph in italic 
+stuff goes here 
+Greetings from Denmark 
<hr>
</BODY>

Syntax highlighting
*.py diff=color
pip install Pygments
For reference:
.gitattributes
config
[diff "color"]
textconv=pygmentize

git diff --cached primes.py
...
@@ -0,0 +1,22 @@
+def primes(n):
+ """
+ Compute the sequence of the first n primes.
+ """
+ p = []
+ if n > 0:
+ p = [2, ]
+ for x in range(1, n):
+ q = p[-1] + 1
+ while True:
+ for y in p:
+ if not (q % y):
+ break
+ else:
+ p.append(q)
Syntax highlighted code in diff's
My brain parses coloured python better
Snippet from:
https://github.com/cbcunc/primer (GPL)

Reformatting
*.md diff=wrap
brew install fmt
For reference:
.gitattributes
config
[diff "wrap"]
textconv=fmt

git diff autostash.md
>"Why can't I pull when I have a dirty workspace, when Mercurial can do this out of the box?"
-I gave the immediate answer that this is just Git's way of protecting the user from possibly
harmful and, more importantly, irreversible changes. Git by default takes a very paranoid
approach to any operations that change dirty files in your file system, when Git itself can't get
you out of those changes again. _This is normally considered a feature_. The known "workaround",
or possible workflow, is to stash any changes before doing a pull (with `git stash save`, and
then unstash them again (`git stash pop`) when done. It seems obvious that it should be easy to
automate this with a git alias, but it turns out that this isn't trivial, as git stash doesn't
fail gracefully when there are no local changes.
+I gave the immediate answer that this is just Git's way of protecting the user from possibly
harmful and, more importantly, irreversible changes. Git by default takes a very paranoid
approach to any operations that change dirty files in your file system, when **Git** itself can't
get you out of those changes again. _This is normally considered a feature_. The known
"workaround", or possible workflow, is to stash any changes before doing a pull (with `git stash
save`, and then unstash them again (`git stash pop`) when done. It seems obvious that it should
be easy to automate this with a git alias but it turns out that this isn't trivial. Git stash
does not fail gracefully when there are no local changes.
.md - Before reformatting

git diff autostash.md
@@ -13,13 +13,13 @@ can do this out of the box?"
I gave the immediate answer that this is just Git's way of protecting
the user from possibly harmful and, more importantly, irreversible
changes. Git by default takes a very paranoid approach to any
-operations that change dirty files in your file system, when Git
+operations that change dirty files in your file system, when **Git**
itself can't get you out of those changes again. _This is normally
considered a feature_. The known "workaround", or possible workflow,
is to stash any changes before doing a pull (with `git stash save`,
and then unstash them again (`git stash pop`) when done. It seems
-obvious that it should be easy to automate this with a git alias,
-but it turns out that this isn't trivial, as git stash doesn't fail
+obvious that it should be easy to automate this with a git alias
+but it turns out that this isn't trivial. Git stash does not fail
gracefully when there are no local changes.
.md - after formatting with fmt

Extract meta data
Define a useful driver:
[diff "exif"]
textconv=exiftool
*.jpg diff=exif
*.jpeg diff=exif
*.png diff=exif
*.gif diff=exif

https://openclipart.org/detail/227445/yoda
Let's resize Yoda

git diff Yoda-800px.png
diff --git a/Yoda-800px.png b/Yoda-800px.png
index fc8ee0f..8ed738c 100644
--- a/Yoda-800px.png
+++ b/Yoda-800px.png
@@ -1,24 +1,26 @@
-File Size : 35 kB
-File Modification Date/Time : 2017:01:27 17:29:06+01:00
+File Size : 85 kB
+File Modification Date/Time : 2017:01:30 18:52:20+01:00
File Type : PNG
-Image Width : 800
-Image Height : 640
+Image Width : 600
+Image Height : 480
Bit Depth : 8
-Color Type : RGB
+Color Type : RGB with Alpha
Much more useful
Note: manually abbreviated for slide
But do I know what changed?

Can we see the difference?
cachetextconv caches the result
[diff "jpg"]
textconv=jp2a --width=80
cachetextconv = true
*.jpg diff=jpg
*.jpeg diff=jpg
Can't quite do holograms yet, but...
brew install jp2a

git add Yoda.jpg
git diff --cached
diff --git a/Yoda.jpg b/Yoda.jpg
new file mode 100644
index 0000000..cf0300a
--- /dev/null
+++ b/Yoda.jpg
@@ -0,0 +1,32 @@
+................................................................................
+......................................''''......................................
+................................,:ldxkkOOkkxdl:,................................
+.............................,lxOOdlllxOOxllldOkxc'.............................
+....',:ccllllollc:,'.......'lkOxllllllxOOxllllllkOkl'.......',:cllooollcc:,'....
+..;okOOkccccclllodxkxoc;'.:xOOOkkxddddkOOkodddxkkOOOx;.';coxkxdollccccclkOOko;..
+....':dOxl;;:::::;;;:ldx:oOxolllloddxxkOOkxxddollllokklcxdl:;;;:::::;:okkd:'....
+.......;xOkd:;::::::::':xOxoxOOOOOOOOOOOOOOOOOOOOOOxokOx;,::::::::;:xOOx;.......
+........'okOOd;;:::::,okOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOkl,:::::;;dOOkl'........
+..........;dOOkl;;:;:xOOOOOOOkkOOOOOOOOOOOOOOOOOOkkOOOOOOOx;;:;;okOkd,..........
+............;okOkd,ckOOOOOkc,..,oOOOOOOOOOOOOOkl,..,lkOOOOOk;;dkOkl,............
+..............';lo;kOOOOOOl......xOOOOOOOOOOOOd......oOOOOOOk;dl;...............
+..................lOOOOOOOk;...'ckOOOOOOOOOOOOk:'...:kOOOOOOOc..................
+..................xOOOOOOOOOxddkOOOOOOOOOOOOOOOOkddkOOOOOOOOOo..................

https://openclipart.org/detail/227445/yoda
Let's give him eyes

diff --git a/Yoda.jpg b/Yoda.jpg
index cf0300a..c16b7de 100644
--- a/Yoda.jpg
+++ b/Yoda.jpg
@@ -9,8 +9,8 @@
........'okOOd;;:::::,okOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOkl,:::::;;dOOkl'........
..........;dOOkl;;:;:xOOOOOOOkkOOOOOOOOOOOOOOOOOOkkOOOOOOOx;;:;;okOkd,..........
............;okOkd,ckOOOOOkc,..,oOOOOOOOOOOOOOkl,..,lkOOOOOk;;dkOkl,............
-..............';lo;kOOOOOOl......xOOOOOOOOOOOOd......oOOOOOOk;dl;...............
-..................lOOOOOOOk;...'ckOOOOOOOOOOOOk:'...:kOOOOOOOc..................
+..............';lo;kOOOOOOl'oxc..xOOOOOOOOOOOOd,dd:..oOOOOOOk;dl;...............
+..................lOOOOOOOk:odc'ckOOOOOOOOOOOOkldd:.:kOOOOOOOc..................
..................xOOOOOOOOOxddkOOOOOOOOOOOOOOOOkddkOOOOOOOOOo..................
..................dOOOOOOOOOOOOOkookOOOOOOOOOdldOOOOOOOOOOOOOl..................
..................,kOOOOOOOOOOkcc:oxkOOOOOOkxocccxOOOOOOOOOOx'..................
git diff Yoda.jpg

Other quick ones
MP3 file meta data: exiftool can do those too, or try mp3info:
[diff "mp3"]
textconv=mp3info -x
Excel files:
[diff "xlsconv"]
textconv=xls2csv
Zip files:
[diff "zipshow"]
textconv=unzip -c -a
[diff "ziplist"]
textconv=unzip -l

Length Date Time Name
--------- ---------- ----- ----
0 01-25-2017 12:52 stuff/
0 01-25-2017 12:53 stuff/foo/
7 01-25-2017 12:53 stuff/foo/bar
12 01-25-2017 12:52 stuff/hello.world
- 9 01-25-2017 12:52 stuff/hi.txt
--------- -------
- 28 5 files
+ 19 4 files
git diff stuff.zip

And we haven't even left Tatooine yet!

Recap - Git objects
https://github.com/pluralsight/git-internals-pdf

Filter drivers
Process blobs on
save or checkout
[filter "name"]
clean = <command>
smudge = <command>
Let's have some fun with them too...

There is no try...
Yapf = Yet Another Python Formatter
( pip install --user yapf )
[filter "cleanpython"]
clean = yapf
smudge = cat
*.py filter=cleanpython

ugly.py
x = { 'a':37,'b':42,
'c':927}
y = 'hello ''world'
z = 'hello '+'world'
a = 'hello {}'.format('world' )
class foo ( object ):
def f (self ):
return 37*-+2
def g(self, x,y=42):
return y
def f ( a ) :
return 37+-+a[42-x : y**3]
git add ugly.py
git commit ....
git push

Caveat
That was a conceptual demo - probably not production ready.
Git diff gets a bit confused:
● git status now shows ugly.py as modified
● git add - and it disappears :-)

“Your eyes can deceive you. Don’t trust them.” – Obi-Wan
For demo: simple rot13 'encryption' - only affects letters
[filter "secret"]
clean = ruby -ne 'print $_.tr( "A-Za-z", "N-ZA-Mn-za-m") '
smudge = perl -pe 'tr/N-ZA-Mn-za-m/A-Za-z/'
*.java filter=secret

public static void sort(int[] numbers) {
int n = numbers.length;
int temp = 0;
for (int i = 0; i < n; i++) {
for (int j = 1; j < (n - i); j++) {
if (numbers[j - 1] > numbers[j]) { //comment
temp = numbers[j - 1];
numbers[j - 1] = numbers[j];
numbers[j] = temp;
}
}
}
}
sort.java

git ls-tree HEAD
100644 blob cf0300a887b362db6e26891afb6ad86757e00f72 Yoda.jpg
100644 blob 60f95e16c59e6e0ef7dc5542cc0f5c15a35e9df4 primes.py
100644 blob 887177cdd3976c695ec5303e6a6d2c1832b458a9 sort.java
100644 blob 809ec4f968232287368681257a0b5ad5c726c08a ugly.py
git show 887177cdd
choyvp fgngvp ibvq fbeg(vag[] ahzoref) {
vag a = ahzoref.yratgu;
vag grzc = 0;
sbe (vag v = 0; v < a; v++) {
sbe (vag w = 1; w < (a - v); w++) {
vs (ahzoref[w - 1] > ahzoref[w]) {
grzc = ahzoref[w - 1];

Now just imagine we had used
● GPG or AES-256 with private key
● *.cred filter=secret
Publish your private content to public repos :-)

The End
"Already know you that which you need." – Yoda

Jedi Mind Tricks in Git

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (10)

Similar to Jedi Mind Tricks in Git

Similar to Jedi Mind Tricks in Git (20)

More from Johan Abildskov

More from Johan Abildskov (8)

Recently uploaded

Recently uploaded (20)

Jedi Mind Tricks in Git