Jdd 2016 DROP DATABASE

Jarosław Ratajski
CSS Versicherung
DROP DATABASE

Jarek Ratajski
Developer, wizard, anarchitectDeveloper, wizard, anarchitect
Lives in LuzernLives in Luzern
Works in CSS InsuranceWorks in CSS Insurance
C64, 6502, 68000, 8086, C++, Java, JEE, Scala, Akka, JS,C64, 6502, 68000, 8086, C++, Java, JEE, Scala, Akka, JS,
ScalaJS....ScalaJS....
jratajski@gmail.comjratajski@gmail.com @jarek000000@jarek000000

10 000 Planets 3 Variants
3 Sizes
HAWAII
MARGHERITTA
VEGETARIAN
MEDIUM
LARGE
XL
Domain summary

MARGHERITTA MEDIUM
HAWAII LARGE
HAWAII LARGE
VEGERARIAN LARGE
HAWAII XL
MARGHERITTA XL
HAWAII LARGE
HAWAII LARGE
HAWAII LARGE
HAWAII LARGE
33
33
11
11
Delivery model

10 000 Pizzas / s
I want to look at number
of queued orders 10000 times / s
My wife also!
Requirements

Alien development
(in 2 „parallel“ universes)

Normal universe Other universe
++

In Normal universe
https://github.com/airomem/galakpizza/tree/master/galakpizza_n

Normal domain
public class Order implements Serializable {
private static final long serialVersionUID = 1L;
public final long id;
public final String planet;
public final Variant variant;
public final Size size;
public Order(long id, String planet, Variant variant, Size size) {
this.id = id;
this.planet = planet;
this.size = size;
this.variant = variant;
}
}

Normal domain 2
public enum Size {
MEDIUM,
LARGE,
XL
}
public enum Variant {
HAWAII,
MARGHERITA,
VEGETARIAN
}

Normal Service
public interface GalakPizzaOrders {
long placeOrder(String planet, Variant variant, Size size);
}

Normal Service
}
public interface GalakPizzaDelivery {
List<Order> takeOrdersFromBestPlanet();
long countStandingOrders();
}

Normal Service
}
}
public interface GalakPizzaService extends GalakPizzaDelivery,
GalakPizzaOrders{
}

Normal service implementation...
public class GalakPizzaCore implements GalakPizzaService, Serializable,
Storable<GalakPizzaCore> {
public long placeOrder(String planet, Variant variant, Size size) {
...
}
public List<Order> takeOrdersFromBestPlanet() {
...
}
public long countStandingOrders() {
...
}
}

private finalprivate final Map<String, List<Order>>Map<String, List<Order>> ordersorders == newnew HashMap<>();HashMap<>();
...
}
...
}
...
}
}

private finalprivate final Map<String, PlanetOrders>Map<String, PlanetOrders> ordersorders == newnew HashMap<>();HashMap<>();
...
}
...
}
...
}
}

PlanetOrders - helper
class PlanetOrders implements Serializable {
final String name;
private List<Order> orders = new ArrayList<>();
public PlanetOrders(String name) {
this.name = name;
}
...
}

final String name;
this.name = name;
}
void assignOrder(final Order order) {
this.orders.add(order);
}
...
}

final String name;
this.name = name;
}
void assignOrder(final Order order) {
this.orders.add(order);
}
public boolean isEmpty() {
return this.orders.isEmpty();
}
...
}

Normal place order
private final Map<String, PlanetOrders> orders = new HashMap<>();
private long orderSequence = 1;
final long id = orderSequence++;
final Order order = new Order(id, planet, variant, size);
assignOrderToPlanet(order);
return id;
}

Normal place order
return id;
}
private void assignOrderToPlanet(Order order) {
final PlanetOrders po = orders.computeIfAbsent(order.planet,
planetName -> new PlanetOrders(planetName));
po.assignOrder(order);
...

Normal place order
private AtomicLong ordersTotal = new AtomicLong(0);
ordersTotal.incrementAndGet();
return id;
}
...

Normal place order revisited
public class GalakPizzaCore { ...
private PriorityQueue<PlanetOrders> bestPlanets = new PriorityQueue<>(256);
return id;
}
this.bestPlanets.offer(po);
}

private PriorityQueue<PlanetOrders> bestPlanets = new PriorityQueue<>(256);
return id;
}
this.bestPlanets.offer(po); BAD BAD BAD BAD
}

private PriorityQueue<PlanetOrders.Wrapper> bestPlanets = new PriorityQueue<>(256);
return id;
}
this.bestPlanets.offer(new PlanetOrders.Wrapper(po));
}

Wrapper... yeah
public static class Wrapper implements Comparable<Wrapper>,Serializable{
final PlanetOrders porders;
final int size; //just keep it final
public Wrapper(PlanetOrders porders) {
this.porders = porders;
this.size = porders.orders.size();
}
@Override
public int compareTo(final Wrapper other) {
return other.size - this.size;
}
}

PlanetOrders – take orders
final String name;
List<Order> takeOrders() {
final List<Order> result = this.orders;
this.orders = new ArrayList<>();
return result;
}
}
}

PlanetOrders – take orders
final String name;
List<Order> takeOrders() {
final List<Order> result = this.orders;
this.orders = new ArrayList<>(); //yes we MUTATE!
return result;
}
}
}

Normal take orders
privateprivate PriorityQueue<PlanetOrders.Wrapper>PriorityQueue<PlanetOrders.Wrapper> bestPlanetsbestPlanets == newnew
PriorityQueue<>(PriorityQueue<>(256256););
final Optional<PlanetOrders> planetOpt = takeBestPlanet();
....
}
private Optional<PlanetOrders> takeBestPlanet() {
PlanetOrders.Wrapper planet = this.bestPlanets.poll();
while (planet != null && planet.porders.isEmpty()) {
planet = this.bestPlanets.poll();
}
return Optional.ofNullable(planet).map(p->p.porders);
}

Normal take orders
if (planetOpt.isPresent()) {
final PlanetOrders planet = planetOpt.get();
List<Order> orders = planet.takeOrders();
return orders;
}
return Collections.EMPTY_LIST;
}
}
}

Normal take orders
if (planetOpt.isPresent()) {
final PlanetOrders planet = planetOpt.get();
List<Order> orders = planet.takeOrders();
ordersTotal.addAndGet(-orders.size());
return orders;
}
}
}
}

Normal count orders
return this.ordersTotal.get();
}

We have service - but do we deal with?
Concurrency
Persistence

Normal - Alien service wrapper
public class GalakPizza implements GalakPizzaService {
final PersistenceController<GalakPizzaCoreGalakPizzaCore,GalakPizzaCore> controller;
public GalakPizza() {
controller = PrevaylerBuilder.newBuilder()
.withinUserFolder("pizza")
.useSupplier( () -> new GalakPizzaCore())
.build();
}
public long placeOrder(final String planet, final Variant variant, final Size size) {
return controller.executeAndQuery( core -> core.placeOrder(planet,variant,size)core.placeOrder(planet,variant,size));
}
return controller.executeAndQuery( core ->core.takeOrdersFromBestPlanet()core.takeOrdersFromBestPlanet());
}
return controller.query( c->c.countStandingOrders()c.countStandingOrders());
}
...
}

In alternative universe
https://github.com/airomem/galakpizza/tree/master/galakpizza_h

Other Domain
@Entity
@Table(name="T_ORDER")
@NamedQueries( {
@NamedQuery(name = "select best planet", query = "SELECT o.planet, count(o) FROM Order o " +
" GROUP BY o.planet ORDER BY count(o) desc"),
@NamedQuery(name = "select orders", query = "SELECT o FROM Order o WHERE o.planet = :planet"),
@NamedQuery(name = "count orders", query = "SELECT count(o) FROM Order o ")
})
public class Order {
@Id
@GeneratedValue(generator="increment")
@GenericGenerator(name="increment", strategy = "increment")
final long id;
public final String planet;
public final Variant variant;
public final Size size;
protected Order() {
this("this is strange...", null, null);
}
public Order(String planet, Variant variant, Size size) {
this.size = size;
id = 0;
}
public long getId() {
return id;
}
}

Other domain (same as above)
public enum Size {
MEDIUM,
LARGE,
XL
}
public enum Variant {
HAWAII,
MARGHERITA,
VEGETARIAN
}

Other service (same)
}
}
public interface GalakPizzaService extends GalakPizzaDelivery,
GalakPizzaOrders{
}

Select best blanet...
SELECT o.planet, count(o) FROM Order o
GROUP BY o.planet ORDER BY count(o) desc

Other Planet (but almost same story)
@Entity
@Table(name="T_PLANET",
indexes = {@Index(name="cnt_idx", columnList = "count", unique = false)} )
@NamedQueries( {
@NamedQuery(name = "select best planet from table", query = "SELECT p FROM Planet p " +
" ORDER BY p.count desc")})
public class Planet {
@Id
public final String name;
private int count;
protected Planet() {
name = "default";
}
public Planet(String name) {
this.name = name;
}
public int getCount() {
return this.count;
}
public void increment() {
this.count++;
}
public void clear() {
this.count = 0;
}
}

Other service implementation
public class GalakPizza implements GalakPizzaService {
final SessionFactory sessionFactory;
public GalakPizza(SessionFactory sessionFactory) {
this.sessionFactory = sessionFactory;
}
...
}
...
}
...
}
}

Other - Place order
return this.runInSession(session -> {
final Order orderEntity = new Order(planet, variant, size);
final Long key = (Long) session.save(orderEntity);
incrementPlanetCounter(planet, session);
return key;
});
}
private void incrementPlanetCounter(String planet, Session session) {
Planet planetEntity = session.get(Planet.class, planet);
if (planetEntity == null) {
planetEntity = new Planet(planet);
session.save(planetEntity);
}
planetEntity.increment();
}
private <T> T runOnSession(Function<Session, T> dbCommand) { ...}

Other - runInSession
private <T> T runInSession(Function<Session, T> dbCommand) {
final Session session = sessionFactory.openSession();
session.beginTransaction();
try {
final T result = dbCommand.apply(session);
session.getTransaction().commit();
session.close();
return result;
} catch (ConstraintViolationException cve) {
session.getTransaction().rollback();
session.close();
return runInSession(dbCommand);
} catch (Throwable t) {
t.printStackTrace();
throw new RuntimeException(t);
}
}
So easy

Other - Take orders
final Query bestPlanetQuery = session.getNamedQuery("SELECT p FROM Planet p " +
" ORDER BY p.count desc");
bestPlanetQuery.setMaxResults(1);
final Iterator<Planet> bestPlanetsIterator = bestPlanetQuery.iterate();
if (bestPlanetsIterator.hasNext()) {
final Planet bestOne = bestPlanetsIterator.next();
final Query ordersQuery = session.getNamedQuery("SELECT o FROM Order o WHERE
o.planet = :planet");
ordersQuery.setParameter("planet", bestOne.name);
final List<Order> orders = ordersQuery.list();
orders.forEach(o -> session.delete(o));
bestOne.clear();
return orders;
}
});
}

Other – count orders
@Override
final Query ordersCount = session.getNamedQuery("count
orders");
final Long cnt = (Long) ordersCount.iterate().next();
return cnt;
});
}

Other - others
Hibernate config, init
MySql config

That we can simulate as
Customers - 4 Threads doing in loop placeOrder (random planet
name, variant and size) – no pauses
PizzBoy - 1 Thread doing in loop takeBestOrder – no pauses
Owners – 2 Threads doing in loop – countStandingOrders

Galaxy simulation
public static void main(String... args) {
runSimulation(5000);
System.gc();
sleep();
Result result = runSimulation(SECONDS_TOTAL*1000);
System.out.println("Orders processed:" + result.processed);
System.out.println("p/s:" + (double)result.processed/(double)
SECONDS_TOTAL);
System.out.println("Orders seen:" + result.seen);
System.out.println("o/s:" + (double)result.seen/(double)
SECONDS_TOTAL);
}

~40 000 p/s ~600 p/s
Processed orders

~ 12 823 500 p/s ~6362 p/s
Peeping speed

GODS of SQL
HELP! ME

~60 000 p/s ~1000 p/s

After few days of operations...

Reality strickes back
I do not trust PizzaBoy
I want to know where exactly were pizzas delivered

Cool, we will not delete
orders any more,
only mark them
as deleted!
Cool, we will add a list
that will be processed
during takeOrders,
or maybe
we will store data
in DB Table (as archive)

Reality strickes back
I do not trust PizzaBoy
I want to know where were pizzas delivered
Yesterday!!!

Crap...
We will restart system
and commands
will be reprocessed

System evolves
I want to let people compose their pizza/
Like: more cheese on one half,
and no meat on the next half,
And onion on the third half.

We will map it with only 20 tables
Or do we add TEXT field?
public Order(long id, String
planet, Supplier<Variant>
pizzaDescription, Size size) {
public Order(long id, String
planet, Variant variant, Size
size) {
We will use factory

Difference
+ Fast
+ Code in Java
+ „Strongly“ typed
+ Flexible
+ Testable
- Sophisticated java
- Terribly slow
- Code in Strings and
Annotations
- Stringly typed
- Limited domain
- Testing expensive
+ Easy java

Sweet spot known as 90's
Computers had mostly one CPU Core
Memory (RAM) was expensive but more as 100MB
available in single unit
Disks were slow but huge (like 2Gb-20Gb)
Time for JEE

Normal universe
RAM is so cheap and huge
Computers have multiple cores
Alien developers start to use full power of hardware with Java
No need to store data in databases anymore
Because RAM is so cheap and huge

Other universe
RAM is cheap and huge
Computers have multiple cores
But alien developers have never realised that

Other universe is a good place
Database vendors earn money and are happy
Application Servers vendors earn money and are happy
Power plants are happy
Hardware vendors are happy
Only business complains – but they still pay
Developers have a lot more work to do – they are happy..
Gfx By pressfoto / Freepik

DB / ORM vs Prevalence Problems
Problem Prevalent (Normal) ORMs (Other)
Detached objects Yes Yes
Transaction
isolation
Yes in JAVA
(concurrency)
Yes in DB
Testing Trivial Awful
Cache It is all cache Lots of configs...
Magic Contained Everywhere
Mapping All serializable goes Limited

If you want to jump to normal universe...

Simple start
Prevayler
or
Airomem (Prevayler wrapper for Java 8 )
or
Nothing :-)

Airomem quick - maven
<dependency>
<groupId>pl.setblack</groupId>
<artifactId>airomem-core</artifactId>
<version>1.0.5</version>
</dependency>

Airomem - init
PersistenceController<GalakPizzaCore,GalakPizzaCore> controller =
controller = PrevaylerBuilder
.newBuilder()
.withinUserFolder("pizza")
.useSupplier( () -> new GalakPizzaCore())
.build();
GalakPizzaCore is main/plain object - Root aggregate!
Must be Serializable! (how bad)

Airomem – commands
controller.executeAndQuery( core->core.placeOrder(planet, variant, size ));
----------------------
controller.executeAndQuery( new PlaceOrderCommand(planet,variant,size));
private static final class PlaceOrderCommand implements Command<GalakPizzaCore, Long>
{
private final String planet;
private final Variant variant;
private final Size size;
public PlaceOrderCommand(String planet, Variant variant, Size size){
this.size = size;
}
public Long execute(GalakPizzaCore system) {
return system.placeOrder(planet,variant, size);
}
}

Commands are never executed concurrently

Query
controller.query( c->c.countStandingOrders());
Queries are executed concurrently - also with
one command!

Airomem – snapshots, events
jarek@ubuntu:~$ cd prevayler/
jarek@ubuntu:~/prevayler$ ls
pizza
jarek@ubuntu:~/prevayler$ cd pizza/
jarek@ubuntu:~/prevayler/pizza$ ls -al
total 188
drwxrwxr-x 2 jarek jarek 4096 May 6 14:46 .
drwxrwxr-x 3 jarek jarek 4096 Apr 26 13:40 ..
-rw-rw-r-- 1 jarek jarek 675 Apr 26 14:17 0000000000000000001.journal
-rw-rw-r-- 1 jarek jarek 3375 Apr 26 14:18 0000000000000000002.journal
-rw-rw-r-- 1 jarek jarek 3369 May 5 10:51 0000000000000000007.journal
-rw-rw-r-- 1 jarek jarek 2048 May 5 14:12 0000000000000000012.snapshot

Data migration
Not big deal in fact
Java Serialization (sth to learn)
readResolve, writeReplace
XML export option

Transaction
This is always transactional!!!
public Long execute(GalakPizzaCore system) {
return system.placeOrder(planet,variant, size);
}

Concurrency
Learn java.util.concurrent, synchronized or die!

Replication/ failover
Not in standard... but
Trivial to implement - send events to other machines, share
folder

Indices
Indices in memory do exists
CQEngine
Query<Car> query1 = or(endsWith(Car.NAME, "vic"),
lessThan(Car.CAR_ID, 2));
for (Car car : cars.retrieve(query1)) {
System.out.println(car);
}

Airomem summary
CQRS + ES simplified - one command one event→
With RAM projection

More enterprisy alternatives
Lagom!
Akka persistence
CQRS generally

Advice
Remember to add Thread.sleep(1000) often

Storing in Prevayler vs DB
model events vs model domain
store facts vs store state
Shared state is a cache vs shared state is a problem
Domain with little magic vs domain polluted
Clean architecture vs Maybe DDD can help...

End
Try on something small first (not bigger then one planet)
Drop DB from core of your system
Drop your application servers
Drop annotations
Drop magic
Learn Java and algorithms
Feel the power of RAM and CPU alltogether!

Jdd 2016 DROP DATABASE

Recommended

Recommended

More Related Content

What's hot

What's hot (19)

Viewers also liked

Viewers also liked (6)

Similar to Jdd 2016 DROP DATABASE

Similar to Jdd 2016 DROP DATABASE (20)

More from Jarek Ratajski

More from Jarek Ratajski (16)

Recently uploaded

Recently uploaded (20)

Jdd 2016 DROP DATABASE