Updated info on Paws, an up to date Perl AWS SDK. Presented in Barcelona Perl Workshop in November 2015

1. Paws – A Perl AWS SDK
@pplu_io
06/11/2015 - Barcelona
Barcelona Perl Workshop 2015
Jose Luis Martinez

2. AWS is…
• Cloud Computing
• Consume computing/database/queuing/etc services via an API
• The biggest datacenter you can think of
• With global reach
• Everything is an API 

3. AWS is…
Programmers wet dream

4. AWS is…
Programmers wet dream

5. AWS is…
Builders wet dream

6. Why?
Isn’t there support for AWS on CPAN?

7. AWS Services on CPAN
• There are a LOT
• EC2, SQS, S3, RDS, DynamoDB, etc
• But lots are were missing
• AWS::CLIWrapper is a generic solution too
• Shells off to the oficial AWS CLI (python)
I want Perl support for ALL of them

8. Different authors, different opinions
• Default region (eu-west-1 for some, us-east-1 for others)
• Different HTTP clients
• LWP, HTTP::Tiny, Furl, etc
I want explicit, required, region. Croak if not specified
Pluggable HTTP client?

11. Different authors, different photo
• Some regions not supported due to bugs
• Subtle name changes in region endpoints
• Credential handling
• Module just covers their needs
I want as broad support as we can get

12. Credential handling
• Roles in AWS help you not have to distribute credentials (AccessKey
and SecretKey)
• Support depends on author of module knowing of them / needing them
I want support for Instance Roles, STS AssumeRole, Federation for all
services

13. UpToDate-ness
• Being up to date depends on authors needs, time, etc
• AWS APIs are updated a lot
I want up to date APIs

14. Lets write an SDK!

16. Some numbers
52 services

17. Some numbers
52 services That was in 2015-09

18. Some numbers
60 services Today (2015-11)

19. Some numbers
60 services
~1600 actions

20. Some numbers
60 services
~1600 actions
~3700 distinct input/output objects

21. Some numbers
60 services
~1600 actions
~3700 distinct input/output objects
~12000 attributes

23. Write by hand?

24. Write by hand?

25. Paws is autogenerated

27. Using Paws

28. Authentication
• Don’t burn credentials in your code (or even your apps config)
Use ENV variables AWS_ACCESS_KEY_ID and
AWS_SECRET_ACCESS_KEY
In file ~/.aws/credentials
[default]
aws_access_key_id=AKIXKDSJFDSFSJD
aws_secret_access_key=Gdksjfkso+erkkdjreiw-t

29. Each AWS API is a “Service Class”
• Each Action in the API is a method on the Service Class
• EC2 API -> Paws::EC2 service class
• Paws::EC2 objects have methods like
• RunInstances
• TerminateInstances
• DescribeInstances

30. How do I get an instance of a service
class?
use Paws;
my $ec2 = Paws->service(‘EC2’, region => ‘eu-west-1’);
my $iam = Paws->service(‘IAM’);
# $ec2 and $iam are instances of Paws::EC2 and
Paws::IAM
# they use Paws default config (they just work )

31. How do I get an instance of a service
class? (II)
my $paws = Paws->new(config => {
region => ‘eu-west-1’,
caller => ‘Paws::Net::LWPCaller’,
credentials => ‘My::Custom::Credential::Provider’
});
my $ec2 = $paws->service(‘EC2’);
# ec2 is bound to region ‘eu-west-1’
# and called with LWP
# and gets it’s credentials from some custom source

32. Calling a method
$ec2->Method1(
Param1 => ‘Something’,
Param2 => 42,
Complex1 => {
x => 1,
y => 2,
z => 3
},
Complex2 => [
{ x => 1, y => 2 },
{ x => 2, y => 3 }
])

33. Calling a method
$ec2->Method1(
Param1 => ‘Something’,
Param2 => 42,
Complex1 => {
x => 1,
y => 2,
z => 3
},
Complex2 => [
{ x => 1, y => 2 },
{ x => 2, y => 3 }
])
Docs tell you that this is a Paws::Service::XXX object, but you don’t have
to instance it !!!
Just pass the attributes and the values as a hashref 

34. Calling a method: maps
• Arbitrary key/value pairs
• Don’t build an object either. Paws will handle it for you
• $ec2->Method1(
Map1 => {
x => 1,
y => 2,
z => 3
});

35. Methods return objects
my $object = $x->Method1(…)
Method1 returns Paws::Service::Method1Result
has ‘X’, has ‘Y’, has ‘Complex’ => (isa => ‘Paws::Service::Complex1’)
$object->X
$object->Complex->Complex1Attribute

36. Tricks

37. Tricks: Preloading classes
If you fork, and you do this in every child:
my $ec2 = Paws->service(‘EC2’)
$ec2->RunInstances(…)
Memory for classes gets consumed in each child process
Paws->preload(‘EC2’)
loads ALL EC2 classes.
Paws->preload(‘EC2’, ‘RunInstances’)
loads classes only for $ec2->RunInstances(…)
https://github.com/pplu/aws-sdk-perl/blob/master/examples/preload.pl

38. Tricks: Immutabilizing classes
• Paws has lots of classes for representing lots of things
• They are not (Moose) immutable (thinking that in short lived scripts
immutability makes them slower)
Paws->default_config->immutable(1);
Just after use Paws;
https://github.com/pplu/aws-sdk-perl/blob/master/examples/mutability.pl

39. Tricks: CLI
• Paws ships with a CLI
paws SERVICE --region xx-east-1 DescribeFoo Arg1 Val1
Uses ARGV::Struct to convey nested datastructures via command line

40. Tricks: ARGV::Struct
paws SVC HashProperty { K1 V1 K2 V2 } …
paws SVC ArrayProperty [ 1 2 3 4 ] …

41. Tricks: open_aws_console
• Opens a browser with the AWS console (using the SignIn service)
• Uses your current credentials (extends a temporary token)

42. Tricks: Changing endpoints
my $predictor = $paws->service('ML', region_rules =>
[ { uri => $endpoint_url } ]);
• Works for any service: SQS, EC2…

43. Tricks: Credential providers
• Default one tries to behave like AWS SDKs
• Environment (AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY)
• File (~/.aws/credentials, an ini file)
• From the metadata service (Instance Roles)
• Your own
• Just attach Role “Paws::Credential” to a class, and get the credentials from
wherever 

44. Tricks: Using Roles
If you’re running on an instance in AWS*:
use Paws;
my $ec2 = Paws->service(‘EC2’, region => ‘eu-west-1’);
$ec2->DescribeInstances;
Just works! Look ma! No credentials!
* With an Instance Role (or Instance Profile)

45. Hot news
New License: Apache 2.0

46. Future

48. Future (hint: help needed and accepted)
• Testing Async stuff
• Retrying
• Some APIs return temporary failures
• Want automatic exponential backoff with jitter -> This is in the works on retry branch
• Paging
• Some APIs return paged results
• Want a “give me all of them”  This is in the works on a fix-iterators branch
• Waiters
• Wait until some condition is met
• Want a call to wait until Instance is in running state
• A lot more: take a look at GitHub issues

49. Future: Waiters
• Wait until some condition is met
• Wait until a DynamoDB table is ready
Wait until an instance is running
• Waiters are specified in jmespath format in metadata
• I want jmespath implemented in Perl. See jmespath.org
• Lets implement jmespath in Perl!

50. Future: DynamoDB as a Document store
A Put Requires this
{ ‘A key’ => { ‘S’ => ‘A string’ },
‘Another Key’ => { ‘N’ => 123 },
‘A String array’ => { ‘SS’ => [ ‘String1’, ‘String2’ ] }
}
A document is more like this
{ ‘A key’ => ‘A string’,
‘Another Key’ => 123,
‘A String array’ => [ ‘String1’, ‘String2’ ]
}
So lets convert them for the user: Issue #41 is for the taking 

51. Future (hint: help needed and accepted)
• Object Oriented results
• $ec2->TerminateInstances(InstanceIds => [ ‘i-12345678’ ])
• $instance->Terminate
• Special properties
• En/Decode base64, URIescape, etc
• Better access to ArrayRef and HashRef like properties
• Use Array and Hash Moose trait for
• Number of elements
• Get element i
• Get list of elements, get list of keys

52. Future (hint: help needed and accepted)
• Refactoring generator clases
• MooseX::DataModel
• Template::Toolkit
• Split Paws into separately instalable modules
• Rinse and Repeat
• For other APIs
• AWS API as a Service

53. Support for APIs
0
5
10
15
20
25
30
35
40
45
50
REST Plain
Types of APIs
Query+XML JSON EC2

54. Support for APIs
0
5
10
15
20
25
30
35
40
45
50
REST Plain
Types of APIs
Query+XML JSON EC2
Implemented

55. Support for APIs
0
5
10
15
20
25
30
35
40
45
50
REST Plain
Types of APIs
Query+XML JSON EC2
Implemented
Need love and testing
S3 not working
Route53?
Lambda?
…

56. If you want to contribute

57. Paws is autogenerated
• AWS has some JSON definition files in their SDKs (data-driven)
• Pick them up to generate classes for:
• Actions
• Inputs to actions (parameters)
• Outputs from actions (outputs)
• HTML documentation -> POD
make gen-classes

59. Code generators
• In builder-lib (not distributed on CPAN)
• Paws::API::Builder
• Paws::API::Builder::EC2
• Paws::API::Builder::query
• Paws::API::Builder::json
• Paws::API::Builder::restjson
• Paws::API::Builder::restxml
• Leaves all auto-generated code in auto-lib (distributed on CPAN)
• Hand-written code is in lib

60. Go build!

61. Fork your heart out
https://github.com/pplu/aws-sdk-perl/
Contact me:
Twitter: @pplu_io
Mail: joseluis.martinez@capside.com
CPAN: JLMARTIN
CAPSiDE
Twitter: @capside
Mail: hello@capside.com