2014/12/11 Node.js 社群分享 Node.js 與 google cloud storage

1. Node.js 與 Google
Cloud Storage
多麼痛的領悟

2. 關於我
• Ian Wu
• 瘋⼈人院院⻑⾧長
• 頑⽪皮⼯工坊 Backend Engineer
• http://blog.ianwu.tw/about-me/

3. Why google cloud storage
• 內建 CDN
• Google Cloud Storage behaves essentially like
a Content Delivery Network (CDN) with no work on
your part because publicly readable objects are, by
default, cached in the Google Cloud Storage network.
• try try see
• try 到死
• 有 USD 500 的 credit

4. OAuth2
• JWT (JSON Web Token)
• Google Cloud console
• credential: service account
• covert p12 > pem
Authentication - Google Cloud Storage — Google Cloud Platform
https://cloud.google.com/storage/docs/authentication#service_accounts

5. OAuth2
• Get token
• payload
{
iss: '460520686343-k6tfn73sentmh0ss5nu67kniorbcta8n@developer.gserviceaccount.com',
scope: 'https://www.googleapis.com/auth/devstorage.full_control',
aud: 'https://accounts.google.com/o/oauth2/token',
exp: 1418280623,
iat: 1418280563
}
• jwt sign
// sign with RSA SHA256
var cert = fs.readFileSync('google_cloud_key.pem'); // get private key
var claim = jwt.sign(payload, cert, {
Get Google JWT token.
https://gist.github.com/onlinemad/28341a343ecde186a410
algorithm: 'RS256'
});

6. OAuth2
• token
{
access_token: 'ya29.2QA9sZg_YtCTGJf1d6Vzxr_4ypioiaIdHJBmgxq6b1HsJuAPODCHnCvt',
token_type: 'Bearer',
expires_in: 3600
}
• 使⽤用 token
headers: { Authorization: 'Bearer ' + token.access_token }

7. Upload URI
• Upload URI, for media upload requests
• upload/storage/v1/b/bucket/o
• Metadata URI, for metadata-only requests:
• storage/v1/b/bucket/o
• APIs Explorer currently supports metadata
requests only.

8. Upload method
• simple
• 就 post 上傳檔案
• multipart（推薦使⽤用）
• 可以連 metadata ⼀一起上傳
• request 某⼀一個版本以上才有⽀支援
• resumable
• 沒⽤用過
• node-youtube-resumable-upload
https://github.com/grayleonard/node-youtube-resumable-upload

9. multipart
• request
var url = 'https://www.googleapis.com/upload/storage/v1/b/yourbucket/o?' +
qs.stringify(querystring);
request.post({
preambleCRLF: true,
postambleCRLF: true,
url: url,
multipart: [
{ 'Content-Type': 'application/json', body: JSON.stringify(metadata) },
{ body: __newFile }
],
headers: { Authorization: 'Bearer ' + token.access_token }
});

10. multipart
• body
{
cacheControl: 'public, max-age=604800',
acl: [{
entity: 'allUsers',
role: 'READER'
}, {
entity: 'project-owners-692227494718',
role: 'OWNER'
}]
}
• query string
• 不能跟 Request body ⼀一起⽤用

11. Directory structure
• ⼀一切都是平的
• 跟 s3 ⼀一樣
• 所以沒有建⽴立 folder 這件事情
• name = foo/bar.jpg;

12. Directory structure
• simple
• /o?name=foo%2Fbar.jpg
• multipart
• body.name = foo/bar.jpg

13. Access URL
• Standard（推薦）
• storage.googleapis.com/<bucket>/<object>
• <bucket>.storage.googleapis.com/<object>
• CNAME
• travel-maps.example.com CNAME c.storage.googleapis.com
• no ssl
• Cookie-based Authentication
• 沒⽤用過

14. Versioning
• 預設是關掉的
➜ ~ gsutil versioning get gs://onlinemad-versioning
gs://onlinemad-versioning: Suspended
➜ ~ gsutil versioning set on gs://onlinemad-versioning
Enabling versioning for gs://onlinemad-versioning/...
➜ ~
• qs + generation
{
"kind": "storage#object",
"id": "onlinemad-dev/uploaded.jpg/1418291876469000",
"selfLink": "https://www.googleapis.com/storage/v1/b/onlinemad-dev/o/uploaded.jpg",
"name": "uploaded.jpg",
"bucket": "onlinemad-dev",
"generation": "1418291876469000",
"metageneration": "1",
"contentType": "image/jpeg",
"updated": “2014-12-11T09:57:56.468Z”,
}

15. ACL
[
{
"entity": "project-owners-460520686343",
"projectTeam": {
"projectNumber": "460520686343",
"team": "owners"
},
"role": "OWNER"
},
{
"entity": "project-editors-460520686343",
"projectTeam": {
"projectNumber": "460520686343",
"team": "editors"
},
"role": "OWNER"
},
{
"entity": "project-viewers-460520686343",
"projectTeam": {
"projectNumber": "460520686343",
"team": "viewers"
},
"role": "READER"
},
{
"entity": "user-00b4903a9745459d3abf193213c0f30d5dea50ee7e3e318007a7edfaecb646e5",
"entityId": "00b4903a9745459d3abf193213c0f30d5dea50ee7e3e318007a7edfaecb646e5",
"role": "OWNER"
}
]

16. ACL
• 我需要 public read
• 所以request.post({
preambleCRLF: true,
postambleCRLF: true,
url: url,
multipart: [{
'Content-Type': 'application/json',
body: JSON.stringify({
name: 'acl_multipart_upload_public_read.jpg',
acl: [{
entity: 'allUsers',
role: 'READER'
}]
})
}, {
body: data
}],
headers: {
Authorization: 'Bearer ' + token.access_token
}
})

17. ACL
➜ ~ gsutil acl get gs://onlinemad-dev/
acl_simple_upload_public_read.jpg
AccessDeniedException: Access denied. Please ensure you
have OWNER permission on gs://onlinemad-dev/
acl_simple_upload_public_read.jpg.

19. 這是 feature 不是 bug
這是 feature 不是 bug
這是 feature 不是 bug

20. ACL

22. ACL
request.post({
preambleCRLF: true,
postambleCRLF: true,
url: url,
multipart: [{
'Content-Type': 'application/json',
body: JSON.stringify({
name: 'acl_multipart_upload_public_read_add_owner.jpg',
acl: [{
entity: 'allUsers',
role: 'READER'
}, {
entity: 'project-owners-460520686343',
role: 'OWNER'
}]
})
}, {
body: data
}],
headers: {
Authorization: 'Bearer ' + token.access_token
}
})

23. 我的領悟

24. 「還沒有⼈人分享 Google Service 時，
請勿輕易嘗試」
– Ian Wu

25. 「當你試了 Google Service 時，
請來分享」
– Ian Wu

26. 謝謝⼤大家