All cloud service providers support seamless cloud automation and management through a REST API architecture allowing for single tasks or complex multi-step orchestrations to be created. REST has become the de facto standard for these cloud interfaces because of its ease of us, communication over HTTP, and wide support of nearly all programming languages and operating systems.
Where do you start? How do you decipher the API documentation? Where do you authenticate? And how do you create cloud resources programmatically?
This presentation walks through the fundamentals of REST, how its invoked through cURL, as well as a live demonstration of the automated provisioning of Oracle Cloud services through cURL/REST.
CNIC Information System with Pakdata Cf In Pakistan
Automating Cloud Operations: Everything You Wanted to Know about cURL and REST
1. MICHIGAN ORACLE USERS SUMMIT 2022
WEDNESDAY OCTOBER 26,2022
1:15PM @W210C
AUTOMATING CLOUD OPERATIONS
EverythingYou Needed to Know about REST and cURL
PRESENTER NAME: AHMED ABOULNAGA
PRESENTERTITLE: TECHNICAL DIRECTOR
2. TABLE OF CONTENTS
Introduction 3
REST and Cloud 6
Introduction to REST 9
Introduction to cURL 23
Creating an Oracle Autonomous Database 30
Figuring Out Authentication and Headers 41
Demo 56
4. ABOUT ME
Ahmed Aboulnaga
Master’s degree in Computer Science from George Mason University
Recent emphasis on cloud,DevOps,middleware,security in current projects
OracleACE Pro, OCE, OCA
Author, Blogger,Presenter
@Ahmed_Aboulnaga
6. CLOUD APIS
All cloud vendors provide some type of API to their services
This allows for programmatic access to cloud services
A basic understanding of cURL, REST, and JSON is helpful
Most cloud providers use the REST architectural style for their APIs
Client REST API Cloud Service
JSON / XML
GET / POST / PUT / DELETE
7. GOAL OFTHIS PRESENTATION
Better understanding of REST and JSON
Understand and interpret REST API documentation
Familiarization with authorization when calling REST APIs
Become capable of automating your cloud operations through REST APIs
Target
Audience
https://globalacademy-eg.com/training-courses/oracle-dba.htm
9. MY USE CASE
What I was trying to do?
Create an Identity Domain in Oracle
Access Manager
OAM 12.2.1.3 had a web-based console
for all OAuth configuration
10. MY USE CASE
What was my challenge?
OAM 12.2.1.4 provided no web-
based interface and only supported
a RESTAPI
https://docs.oracle.com/en/middleware/idm/access-manager/12.2.1.3/oroau/op-oam-services-rest-ssa-api-v1-oauthpolicyadmin-oauthidentitydomain-post.html
11. STARTWITHTHE DOCUMENTATION
Was the documentation helpful?
Not all RESTAPI documentation is
created equally
Fortunately,the OAM REST API
documentation provided an example
request and example response
12. PREPARINGTO LOGIN
What did I do first?
Passwords can be passed as a
“username:password” combination or
encoded
Encoding converts literal text to a humanly
unreadable format
Used online to encode“weblogic:welcome1”
Authentication Options:
curl -u 'weblogic':'welcome1'
curl -H 'Authorization:Basic d2VibG9naWM6d2VsY29tZTE='
https://www.base64encode.org
13. DID ITWORK?
Initial invocation failed command:
curl -H 'Content-Type: application/x-www-form-urlencoded'
-H 'Authorization:Basic d1VibG9naWM6d2VsY29tZTE='
--request POST
http://soadev.revtech.com:7701/oam/services/rest/ssa/api/v1/oauth
policyadmin/oauthidentitydomain
-d ' {
"name" : "AhmedWebGateDomain",
"identityProvider" : "AhmedOUDStore",
"description" : "Ahmed OIDC Domain"
} '
Output:
Mandatory param not found. Entity - IdentityDomain, paramName - name
The documentation states that only “name” is mandatory
So what’s the problem?
14. WHATWASTHE SUCCESSFUL OUTCOME?
Final successful command:
curl -H 'Content-Type: application/x-www-form-urlencoded' -H 'Authorization:Basic
d1VibG9naWM6d2VsY29tZTE='
'http://soadev.revtech.com:7701/oam/services/rest/ssa/api/v1/oauthpolicyadmin/oauthidentitydomai
n' -d '{"name":"AhmedWebGateDomain", "identityProvider":"AhmedOUDStore", "description":"Ahmed
OIDC WebGate Domain",
"tokenSettings":[{"tokenType":"ACCESS_TOKEN","tokenExpiry":3600,"lifeCycleEnabled":false,"refres
hTokenEnabled":false,"refreshTokenExpiry":86400,"refreshTokenLifeCycleEnabled":false},
{"tokenType":"AUTHZ_CODE","tokenExpiry":3600,"lifeCycleEnabled":false,"refreshTokenEnabled":fals
e,"refreshTokenExpiry":86400,"refreshTokenLifeCycleEnabled":false},
{"tokenType":"SSO_LINK_TOKEN","tokenExpiry":3600,"lifeCycleEnabled":false,"refreshTokenEnabled":
false,"refreshTokenExpiry":86400,"refreshTokenLifeCycleEnabled":false}],
"errorPageURL":"/oam/pages/servererror.jsp", "consentPageURL":"/oam/pages/consent.jsp"}'
Impossible to determine accurate command without support,examples,or thorough documentation
Required Oracle Support assistance to get these details
16. WHAT IS REST?
REpresentational StateTransfer
Architectural style for distributed hypermedia system
Proposed in 2000 by Roy Fielding in his dissertation
Web Service implemented with REST is called RESTful web service
REST is not a protocol like SOAP.It is rather an architectural style
REST services typically use HTTP/HTTPS,but can be implemented with other protocols like FTP
17. REST ARCHITECTURAL CONSIDERATIONS
Uniform interface: Easy to understand and readable results and can be
consumed by any client or programming language over basic protocols.
URI-based access: Using the same approach to a human browsing a
website where all resource are linked together.
Stateless communication: Extremely scalable since no client context is
stored on the server between requests.
18. REST METHODS
The HTTP protocol provides multiple methods which you can utilize for RESTful web services
The table maps the HTTP method to the typical REST operation
Some firewalls may limit some HTTP methods for security reasons
HTTP Method REST Operation
GET Read
POST Create
PUT Update
DELETE Delete
OPTIONS List of available methods
HEAD Get version
PATCH Update property/attribute
Most common in
web applications
Most common in
REST to provide
CRUD functionality
19. RESOURCES
Requests are sent to resources (i.e., URLs)
Each resource represents an object which identified by a noun (e.g., employee,etc.)
Each resource has a unique URL
When performing a POST (create) or PUT (update),you must pass additional values
Resource HTTP Method REST Output
https://hostname/hr/employee GET Retrieve a list of all employees
https://hostname/hr/employee/12 GET Retrieve details for employee #12
https://hostname/hr/employee POST Create a new employee
https://hostname/hr/employee/12 PUT Update employee #12
https://hostname/hr/employee/12 DELETE Delete employee #12
https://hostname/hr/employee/12/address GET Retrieve address for employee #12
20. HTTP RESPONSE CODES
HTTP response codes determine the overall response of the REST invocation
HTTP Code Status Description
2XX (200,201,204) OK Data was received and operation was performed
3XX (301,302) Redirect Request redirected to another URL
4XX (403,404) Client Error Resource not available to client
5XX (500) Server Error Server error
21. JSON
JavaScript Object Notation
Pronounced“Jason”
An object surrounded by { }
An array or ordered list
REST can support both JSON and XML
Less verbose than XML,but lacks metadata support
//JSON Object
{
"employee": {
"id": 12,
"name": "Kobe",
"location": "USA"
}
}
//JSON Array
{
"employees": [
{
"id": 12,
"name": "Kobe",
"location": "USA"
},
{
"id": 13,
"name": "Jordan",
"location": "Canada"
},
{
"id": 14,
"name": "Barkley",
"location": "USA"
}
]
}
23. WHAT IS CURL?
Open-source command-line tool
Supports more than 22 different protocols (e.g.,
HTTP,HTTPS,FTP,etc.)
For HTTP,supports all methods (e.g., GET, POST,
PUT,DELETE, etc.)
Very useful for testing RESTful web services
Other advanced tools available include Postman,
SoapUI,Oracle SQL Developer,etc.
Example service:
https://api.weather.gov/alerts/active?area=MI
26. POSTMAN
PopularAPI client
Free version available
www.postman.com
Numerous features that include:
‒ Create API documentation
‒ Automated testing
‒ Design and mock APIs
‒ MonitorAPIs
‒ Etc.
27. BENEFITS OF CURL
Free
Command-line based tool
Useful for non-interactive scripts
Can pass HTTP headers,cookies,and authentication information
Support for SSL, proxy,numerous protocols (e.g., LDAP, SMB,SCP,IMAP,FILE,TELNET,etc.), etc.
34. NAVIGATETO DOCUMENTATION
CreateAutonomousDatabase Reference
https://docs.cloud.oracle.com/en-us/iaas/api/#/en/database/20160918/AutonomousDatabase/CreateAutonomousDatabase
Note:
‒ API reference
‒ CreateAutonomousD
atabase operation
‒ REST API endpoint
‒ API version
35. VIEW RESOURCE DETAILS AND EXAMPLE
The resource details provides a
list of all required parameters,
often beyond what is
demonstrated in the example
Use the example as a starting
point
{
"compartmentId" : "ocid1.tenancy.oc1..d6cpxn…dbx",
"displayName" : "MOUS DB 2020 Auto",
"dbName" : "MOUSDBAUTO",
"adminPassword" : "Kobe_24_24_24",
"cpuCoreCount" : 1,
"dataStorageSizeInTBs" : 1
}
36. FIRST ATTEMPT… FAILED!
Unable to authenticate upon first try despite all parameters/settings correct per the documentation…
41. DISCOVERY IS PAINFUL
API References and Endpoints
https://docs.cloud.oracle.com/en-
us/iaas/api/#/en/database/20160918/
Oracle Cloud Infrastructure
Documentation
https://docs.cloud.oracle.com/en-
us/iaas/Content/API/Concepts/apisigningkey.htm#How
Managing Autonomous Data
Warehouse Using oci-curl
https://blogs.oracle.com/datawarehousing/managing-
autonomous-data-warehouse-using-oci-curl
Oracle Cloud Infrastructure
(OCI) REST call
walkthrough with curl
https://www.ateam-oracle.com/oracle-cloud-
infrastructure-oci-rest-call-walkthrough-with-curl
But why didn’t
the docs point me
to this?
Found this on my
own, has some
helpful info… I don’t want
to use “oci-
curl”!
This is
complicated!
Thank God they
have a script!
Blog?
Another
blog?
42. PIECING IT TOGETHER
If you want to use cURL to invoke OCI REST APIs…
1. Get information from the OCI Console
a. Get theTenancy ID
b. Get the User ID
2. Generate and configure an API Signing Key
a. Create public/private key pair
b. Get the fingerprint of the key
c. Get the public key from the private key in PEM format
d. Add the API Key to the OCI user (by uploading the public key)
3. Prepare and execute script
a. Ensure private key is available
b. Create the JSON request
c. Update the custom script
d. Execute!
45. 2A. CREATE PUBLIC/PRIVATE KEY PAIR
Use ssh-keygen to create a public/private key pair
The public key will be added as an “API Key” to your OCI account
The private key will be used by your client (i.e., cURL)
46. 2B. GET THE FINGERPRINT OFTHE KEY
Use openssl to view the X.509 MD5 PEM certificate fingerprint
47. 2C. GET PUBLIC KEY FROMTHE PRIVATE KEY IN PEM FORMAT
OCI requires that the public key is imported in PEM format
Use openssl to get the public key in PEM format
48. 2D.ADDTHE API KEY TOTHE OCI USER
The public key is added to the OCI user’sAPI Key
Must be in PEM format
Can be uploaded or pasted
49. 3A. ENSURE PRIVATE KEY IS AVAILABLE
The private key created earlier (and in PEM format) is used when invoking the REST service
50. 3B. CREATE THE JSON REQUEST
The payload is created in JSON format
51. 3C. UPDATE CUSTOM SCRIPT
The various elements (tenancy id, user id,
private key, key fingerprint,etc.) are
parameterized
OCI’s REST API requires additional
calculated elements,all taken care of here
(oci-curl takes care of all of this for you)
The cURL command is eventually called
using a combination of static and dynamic
values
52. 3D. EXECUTE!
The cURL command is
expanded
The cURL command is
executed
The HTTP status code is
observed to obtain the result
of the invocation
55. www.mous.us
THANKYOU
SAVE THE DATE
• ASCEND CONFERENCE 2023
June 11-14,2023
Caribe Royale Resort
Orlando,Florida
https://ascendusersconference.com
• MOUS 2023
October 25, 2023
Schoolcraft College -VisTaTech Center,
18600 Haggerty Rd, Livonia,MI
https://www.mous.us