Asynchronous Systems with Fn Flow

#ASFnFlow @JosePaumard @delabassee
Asynchronous Systems
with Fn Flow

@JosePaumard
@JosePaumard
https://github.com/JosePaumard
https://www.slideshare.net/jpaumard
https://www.youtube.com/user/JPaumard

Questions?
#ASFnFlow

@JosePaumard @delabassee#ASFnFlow
Agenda
Asynchronous Systems with Fn Flow

An Open Source Container Native FaaS Platform
https://github.com/fnproject

Fn
Open Source
Approachable
Container based
Language independent
Platform independent
Scheduler independent
No lock-in
Easy for new users
More controls for advanced users
Leverages Docker
Go, Node, Java, etc.
Cloud, On-Perm, laptop
K8S, etc.
https://github.com/fnproject

Fn Architecture
Fn CLI
Fn FDK’s
Fn Flow

Setting up Fn

Setting Up Fn
1) Works on Linux, Windows and Mac

Setting Up Fn
2) Requires a working Docker installation
▪ Available on Windows Pro but…!
▪ Requires Docker 17.10+
▪ It not, check:
▪ 7 steps to follow to get a fresh Docker installation
$ docker –v
Docker version 18.06.1-ce, build e68fc7a
https://docs.docker.com/install/linux/docker-ce/ubuntu/#prerequisites

Setting Up Fn
3) Execute the install script, directly from GitHub!
▪ After some time:
$ curl -LSs https://raw.githubusercontent.com/fnproject/cli/master/install | sh
fn version 0.5.13
______
/ ____/___
/ /_ / __
/ __/ / / / /
/_/ /_/ /_/`

Setting Up Fn
4) Launch Fn Server
$ fn start

Setting Up Fn
4) Launch Fn Server
You need to be root for most of the Fn commands (*)
$ fn start
docker: Got permission denied while trying to connect to the Docker daemon socket at
unix:///var/run/docker.sock: Post
http://%2Fvar%2Frun%2Fdocker.sock/v1.38/containers/create?name=fnserver: dial unix
/var/run/docker.sock: connect: permission denied.
See 'docker run --help'.
2018/10/04 10:30:47 Error: processed finished with error exit status 126

Setting Up Fn
5) Let us try it again
# fn start

Setting Up Fn
# fn start
Unable to find image 'fnproject/fnserver:latest' locally
latest: Pulling from fnproject/fnserver
ff3a5c916c92: Pull complete
1a649ea86bca: Pull complete
ce35f4d5f86a: Pull complete
b6206661264b: Pull complete
b8b71dba24d3: Pull complete
3873004a68ee: Pull complete
f4205b132661: Pull complete
91a85eeeb257: Pull complete
93c96d032b32: Pull complete
bb761748d6e1: Pull complete
81f6c51c4ac2: Pull complete
2ba715696dba: Pull complete
f46c2b56aaf3: Pull complete
08d1ae9a5cf6: Pull complete
abd91abc85a7: Pull complete

Setting Up Fn
# fn start
Digest: sha256:0c2f52b15f7bde26e6d9e4117be7b7b94e72feeec8da6f12f6dc7eae294a2ea8
Status: Downloaded newer image for fnproject/fnserver:latest
time="2018-10-04T17:37:58Z" level=info msg="Registering container driver 'docker'"
time="2018-10-04T17:37:58Z" level=info msg="Registering log provider 's3'"
time="2018-10-04T17:37:58Z" level=info msg="Registering data store provider 'sql'"
time="2018-10-04T17:37:58Z" level=info msg="Registering log provider 'sql'"
time="2018-10-04T17:37:58Z" level=info msg="Registering sql helper 'mysql'"
time="2018-10-04T17:37:58Z" level=info msg="Registering sql helper 'postgres'"
time="2018-10-04T17:37:58Z" level=info msg="Registering sql helper 'sqlite'"
time="2018-10-04T17:37:58Z" level=info msg="Setting log level to" level=info
time="2018-10-04T17:37:58Z" level=info msg="mysql does not support sqlite3"
time="2018-10-04T17:37:58Z" level=info msg="postgres does not support sqlite3"
time="2018-10-04T17:37:58Z" level=info msg="mysql does not support sqlite3"
time="2018-10-04T17:37:58Z" level=info msg="postgres does not support sqlite3"
time="2018-10-04T17:37:58Z" level=info msg="Connecting to DB" url=/app/data/fn.db
time="2018-10-04T17:37:58Z" level=info msg="datastore dialed" datastore=sqlite3 max_idle_connections=256 url="sqlite3:///app/data/fn.db"
time="2018-10-04T17:37:58Z" level=info msg="agent starting cfg={MinDockerVersion:17.10.0-ce DockerNetworks: DockerLoadFile:

Setting Up Fn
# fn start
FreezeIdle:50ms HotPoll:200ms HotLauncherTimeout:1h0m0s AsyncChewPoll:1m0s MaxResponseSize:0 MaxLogSize:1048576 MaxTotalCPU:0
MaxTotalMemory:0 MaxFsSize:0 PreForkPoolSize:0 PreForkImage:busybox PreForkCmd:tail -f /dev/null PreForkUseOnce:0 PreForkNetworks:
EnableNBResourceTracker:false MaxTmpFsInodes:0 DisableReadOnlyRootFs:false DisableTini:false DisableDebugUserLogs:false
IOFSEnableTmpfs:false IOFSAgentPath:/iofs IOFSMountRoot:/home/ubuntu/.fn/iofs IOFSOpts:}"
time="2018-10-04T17:37:58Z" level=info msg="no docker auths from config files found (this is fine)" error="open /root/.dockercfg: no such
file or directory"
time="2018-10-04T17:37:58Z" level=info msg="available memory" cgroupLimit=9223372036854771712 headRoom=268435456 totalMemory=2468630528
time="2018-10-04T17:37:58Z" level=info msg="ram reservations" availMemory=2200195072 ramAsyncHWMark=1760156057
time="2018-10-04T17:37:58Z" level=info msg="available cpu" availCPU=2000 totalCPU=2000
time="2018-10-04T17:37:58Z" level=info msg="cpu reservations" cpu=2000 cpuAsyncHWMark=1600

Setting Up Fn
5) Let us try again
# fn start
______
/ ____/___
/ /_ / __
/ __/ / / / /
/_/ /_/ /_/
v0.3.591

Setting Up Fn
6) Check your setup
$ fn version
Client version: 0.5.15 is not latest: 0.5.18
Server version: 0.3.591

Fn Concepts

Introducing Fn Function
Function wrapped in a Container Image
Input from stdin
Output to stdout
Logs to stderr
Or simply use an FDK!
Fn handles everything else!

Fn Concepts
In Fn, a function is the fundamental unit
A function is created from (Java) code
A function lives in an « app »
To bootstrap a function:
$ fn init --runtime java myfunction
Creating function at: /myfunction
Function boilerplate generated.
func.yaml created
$ ls myfunction/
func.yaml pom.xml src/

Fn Concepts
Functions are deployed within an « app »
$ fn create app first-app
Successfully created app: first-app
$ fn list apps
NAME
first-app
$ fn delete app first-app
App first-app deleted

Fn Concepts
Fn deals with triggers
A function is accessible through a trigger
# fn list triggers first-app
FUNCTION NAME TYPE SOURCE ENDPOINT
primefunction prime-trigger http /is-prime http://localhost:8080/t/fn-primes/is-prime

Setting up a Fn Java Project

Setting Up a Function
Using the Fn CLI
$ fn init --runtime java --trigger http javafunc
Creating function at: /firstfunc
func.yaml created.

Setting Up a Function
Using the Fn CLI
This command creates a Maven project
With a simple Java class
And a func.yaml file
$ fn init --runtime java --trigger http javafunc
Creating function at: /javafunc
func.yaml created.

The func.yaml file
Function metadatas
https://github.com/fnproject/docs/blob/master/fn/develop/func-file.md

The func.yaml file
Function metadatas
schema_version: 20180708
name: firstfunction
version: 0.0.8
runtime: java
build_image: fnproject/fn-java-fdk-build:jdk9-1.0.72
run_image: fnproject/fn-java-fdk:jdk9-1.0.72
cmd: org.paumard.oc2018.firstFunction.Echo::echo
format: http-stream
memory: 128
timeout: 30
triggers:
- name: firstfunction
type: http
source: /firstfunction

The pom.xml file
The right versions:
<properties>
<maven.compiler.source>9</maven.compiler.source>
<maven.compiler.target>9</maven.compiler.target>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<fdk.version>1.0.72</fdk.version>
</properties>

The pom.xml file
The dependencies:
<dependency>
<groupId>com.fnproject.fn</groupId>
<artifactId>api</artifactId>
<version>${fdk.version}</version>
</dependency>
<dependency>
<groupId>com.fnproject.fn</groupId>
<artifactId>flow-runtime</artifactId>
<version>${fdk.version}</version>
</dependency>

The pom.xml file
The repository:
<repository>
<id>fn-release-repo</id>
<url>https://dl.bintray.com/fnproject/fnproject</url>
<releases>
<enabled>true</enabled>
</releases>
<snapshots>
<enabled>false</enabled>
</snapshots>
</repository>

One last point
Fn is building the function image in a separate container
with its own Maven
configured with access to its own repository

A first function
package org.paumard.oc2018.firstFunction;
public class Echo {
public String echo(String input) {
String name = (input == null || input.isEmpty()) ? "world" : input;
return "Hello, " + name + "!";
}
}

Deploying your first function
Fn CLI takes care of the function deployment
# fn deploy –-app first-app --local firstfunction

Deploying firstfunction to app: first-app
Bumped to version 0.0.53
Building image fndemouser/firstfunction:0.0.53 .....
Updating function firstfunction using image fndemouser/first-app:0.0.53...

This deployment may take some time…
Because Fn is doing many things under the hood
Updating function firstfunction using image fndemouser/first-app:0.0.53...

Let us run in verbose mode
# fn --verbose deploy –-app first-app --local firstfunction
FN_REGISTRY: fndemouser
Current Context: default
Sending build context to Docker daemon 22.02kB
Step 1/11 : FROM fnproject/fn-java-fdk-build:jdk9-1.0.72 as build-stage
---> b6ccfda3fe63
Step 2/11 : WORKDIR /function
---> Using cache
---> c17fc8ffe44c
Step 3/11 : ENV MAVEN_OPTS -Dhttp.proxyHost= -Dhttp.proxyPort= -Dhttps.proxyHost= -
Dhttps.proxyPort= -Dhttp.nonProxyHosts= -Dmaven.repo.local=/usr/share/maven/ref/repository
---> Using cache
---> 8fc66a3f7245
842bed937b9

Step 4/11 : ADD pom.xml /function/pom.xml
---> Using cache
---> 1cdf0a511da1
Step 5/11 : RUN ["mvn", "package", "dependency:copy-dependencies", "-DincludeScope=runtime", "-
DskipTests=true", "-Dmdep.prependGroupId=true", "-DoutputDirectory=target", "--fail-never"]
---> Using cache
---> 3c9c6b613843
Step 6/11 : ADD src /function/src
---> Using cache
---> 8842bed937b9
Step 7/11 : RUN ["mvn", "package"]
---> Running in 27c78d8e9bd3

[INFO] Scanning for projects...
[INFO]
[INFO] ---------------------< org.paumard:main-function >----------------------
[INFO] Building main-function 1.0-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ main-function ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 0 resource
[INFO]
[INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ main-function ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 2 source files to /function/target/classes
[INFO]
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ main-function ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /function/src/test/resources
[INFO]
[INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @ main-function ---
[INFO] Nothing to compile - all classes are up to date
[INFO]
[INFO] --- maven-surefire-plugin:2.12.4:test (default-test) @ main-function ---
[INFO] No tests to run.
[INFO]
[INFO] --- maven-jar-plugin:2.4:jar (default-jar) @ main-function ---
[INFO] Building jar: /function/target/main-function-1.0-SNAPSHOT.jar
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 42.379 s
[INFO] Finished at: 2018-10-18T14:33:09Z
[INFO] ------------------------------------------------------------------------

Removing intermediate container 27c78d8e9bd3
---> 85e15d82caf4
Step 8/11 : FROM fnproject/fn-java-fdk:jdk9-1.0.72
---> d23b2ab9ca0a
Step 9/11 : WORKDIR /function
---> Using cache
---> 6172132f0ccb
Step 10/11 : COPY --from=build-stage /function/target/*.jar /function/app/
---> cc5f53168459
Step 11/11 : CMD ["org.paumard.oc2018.firstFunction.Echo::echo"]
---> Running in 094ae9918e3e
Removing intermediate container 094ae9918e3e
---> e2e55466ff99
Successfully built e2e55466ff99
Successfully tagged fndemouser/firstfunction:0.0.56

Checking the deployment

Check the apps
$ fn list apps
NAME
fn-primes
myapp
travel-booking

Check the apps
Check the functions
$ fn list apps
NAME
fn-primes
myapp
travel-booking
$ fn list functions fn-primes
NAME IMAGE
primefunction fndemouser/primefunction:0.0.36

Check the triggers
$ fn list triggers fn-primes
FUNCTION NAME TYPE SOURCE ENDPOINT
primefunction prime-trigger http /is-prime http://localhost:8080/t/fn-primes/is-prime

Inspect the function
$ fn inspect function travel-booking quotation
{
"annotations": {
"fnproject.io/fn/invokeEndpoint":
"http://localhost:8080/invoke/01CT168S17NG8G00GZJ0000076"
},
"app_id": "01CT152R7RNG8G00GZJ000006A",
"created_at": "2018-10-17T14:18:29.287Z",
"format": "http-stream",
"id": "01CT168S17NG8G00GZJ0000076",
"idle_timeout": 30,
"image": "fndemouser/quotation:0.0.13",
"memory": 128,
"name": "quotation",
"timeout": 30,
"updated_at": "2018-10-21T16:35:07.894Z"
}

Invoke the function
$ fn invoke fn-primes prime-function
…

Function Client
Writing a function client is a matter of writing an HTTP client
We have a nice API for that in Java 11 (*)
https://docs.oracle.com/en/java/javase/11/docs/api/java.net.http/java/net/http/HttpClient.html

Function Client
public static void main(String[] args) throws IOException, InterruptedException {
HttpClient httpClient = HttpClient.newHttpClient();
HttpRequest.BodyPublisher bodyPublisher = HttpRequest.BodyPublishers.ofString("3");
HttpRequest request =
HttpRequest.newBuilder()
.POST(bodyPublisher)
.uri(URI.create("http://localhost:8080/t/fn-primes/is-prime"))
.build();
HttpResponse.BodyHandler<Stream<String>> responseBodyHandler =
HttpResponse.BodyHandlers.ofLines();
HttpResponse<Stream<String>> response = httpClient.send(request, responseBodyHandler);
List<String> list = response.body().collect(Collectors.toList());
System.out.println("list = " + list);
}

Fn Flow

Fn Flow
ts
10
5
5 5
5
8 7
7 8
10

Fn Flow
• For long-running, reliable, scalable functions with
primitives for fork-join, chaining, delays & error handling
• Java support « based » on Java CompletableFuture API
• Additonal languages support on the way
• Work in progress

A simple Flow example

But before…
It needs a little config!
1) Launch the Flow Server
2) Configure the « app » to connect to the Flow Server
# FNSERVER_IP=$(docker inspect --type container -f '{{.NetworkSettings.IPAddress}}' fnserver)
# docker run --rm -p 8081:8081 -d -e API_URL="http://$FNSERVER_IP:8080/invoke"
-e LOG_LEVEL=debug -e no_proxy=$FNSERVER_IP
--name flowserver fnproject/flow:latest
# FLOWSERVER_IP=$(docker inspect --type container -f '{{.NetworkSettings.IPAddress}}' flowserver)
# fn config app fn-primes COMPLETER_BASE_URL "http://$FLOWSERVER_IP:8081"

A real Flow example

A Travel Agency

Resources
• https://github.com/JosePaumard/fn-demo-oracle-code-one-2018
• https://github.com/fnproject/tutorials/blob/master/JavaFDKIntroduction
• https://fnproject.slack.com/

Merci!

Questions?

Asynchronous Systems with Fn Flow

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Ähnlich wie Asynchronous Systems with Fn Flow

Ähnlich wie Asynchronous Systems with Fn Flow (20)

Mehr von José Paumard

Mehr von José Paumard (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Asynchronous Systems with Fn Flow