7. Chmouel Boudjnah - @chmouel - eNovance
• Object Storage
• Not a file system
• Eventually Consistent
• HTTP REST API
• Very Scalable
• Software defined Storage
What is Swift
8. Chmouel Boudjnah - @chmouel - eNovance
Why would you want to use GIT with Swift?
10. Chmouel Boudjnah - @chmouel - eNovance
Why Swift with GIT
When you use a file system you have to :
Perform backups of your repositories regularly
Perform complex and risky operations to extend the storage capabilities
for you repositories.
You have mostly have to manage it yourself
It’s an another piece in your infrastructure
You have to manage the downtime
Not fun
12. Chmouel Boudjnah - @chmouel - eNovance
Why Swift with GIT
The advantages of using Swift
Safety: The inherent replication of a data store.
Capability: easily extend storage capacity.
High Availability: no single point of failure.
Maintainability: allow operations teams to maintain the cluster without
any downtime.
In an OpenStack environment it’s usually already there.
16. Chmouel Boudjnah - @chmouel - eNovance
So how did we plug that to Swift?
17. Chmouel Boudjnah - @chmouel - eNovance
We did not want to modify the GIT client binaries.
18. Chmouel Boudjnah - @chmouel - eNovance
We used a pure python GIT library called Dulwich
19. Chmouel Boudjnah - @chmouel - eNovance
• Create, read, manage loose objects (blob, tree,
commit, tag).
• Create, read, manage pack files, pack indexes.
• Implement the Git smart protocol through git-
upload-pack and git-receive-pack.
• Implement the Git, HTTP, SSH listeners to start
Dulwich as a Git server.
• Provide a well designed interface to the storage
backend. (Repo, MemoryRepo).
Dulwich
20. Chmouel Boudjnah - @chmouel - eNovance
How do we use Dulwhich
We implemented a custom repo called SwiftRepo as a backend to the git
listener of Dulwich which communicate to OpenStack Swift
21. Chmouel Boudjnah - @chmouel - eNovance
So about that SwiftRepo backend how does it
work?
22. Chmouel Boudjnah - @chmouel - eNovance
SwiftRepos backend to Dulwich
It doesn’t use the standard way to store Git
references (one file by reference).
It doesn’t store loose objects but instead uses the pack format.
In addition to the .index that relate to each pack file we added .info object
It makes the Request to Swift concurrently.