SlideShare ist ein Scribd-Unternehmen logo
1 von 49
Downloaden Sie, um offline zu lesen
On the integrity of data in Java
Applications
Lucas Jellema (AMIS)
NLJUG JFall 2013
6th November 2013, Nijkerk, The Netherlands
Agenda
• What is integrity?
• Enforcing data constraints
– throughout the application architecture
• Transactions
• Exclusive Access to …
• The Distributed World
3

Definition of Integrity
• Truth
– Nothing but the truth

• The Only Truth
• [Degree of] success or
completeness of
actions is known
4

Sufficient Integrity

Integrity

Integrity
7,0
48,23

π
Corruptible
Corrupted

Uncorrupted

33,0000002
“five”

42
Correct
Complete
Consistent
Reliable
5

Conference Application
6

Conference Application

Client
(HTML 5 & Java Script)

Web Tier
JavaServer Faces
POJO
Domain
Model

Business Tier
JPA

RDBMS

EJB
7

Validation at entry time
8

Validation at entry time
Client and View
9

Validation at entry time
Client and View
More validation at entry time –
bean Validation

10
11

Validation at entry time
Bean Validation in View
13

Record (Type) level rules
• Program should be
Kids when age < 18;
and Management or
Developer when age > 18
• Using JavaScript
– when either field changes
(handle nulls)
– on submit of the entire
record

• Using Bean Validation:
custom type validator
– in either web-tier or JPA
14

Type Level Constraints with
Bean Validation
15

Type Level Bean Validation:
Custom Validator
16

Validation Implementation
options & considerations
Native
Mobile Client

Native HTML 5;
JavaScript
Client
(pure HTML 5 & Java
Script)

Native HTML 5;
JavaScript
Client
(JSF based HTML 5 & Java Script)
Custom;
Web Tier JSF Validator;
Bean
JavaServer Faces Validation

Custom;
Bean Validation
RESTful Services
POJO
Domain
Model

Business Tier
JPA

RDBMS

EJB
Custom;
Bean Validation
17

But wait – there is more!
• More User Interfaces
• More Attendee
•
•
•

•

Instances
More Entities
& More types
of Constraints
More Users, Sessions,
and Transactions
More Nodes in
the Middle Tier Cluster
More Data Stores
18

Domain model
•
•
•
•
•
•

Attendee
Speaker
Session
Room
Slot
Attendance
– Booked
– Realized
19

Multiple-Instances-of-Single-Entity
constraints
• Constraints that cover multiple same type objects/instances
–
–
–
–
–
–

Attendee‟s Registration Id is unique
No more than 5 conference attendees from the same company
Not more than two sessions by the same speaker
At most one session scheduled per room per slot
Only one keynote session in a slot
Sessions from up to a maximum of three tracks can be scheduled in the same room
20

Inter entity constraints
• Attendees can only attend one hands-on session during the conference
• A person cannot attend another session in a slot in which the session
(s)he is speaker of is scheduled
• No more planned session attendances are allowed than the capacity of
the room in which the session is scheduled to take place
• If the room capacity is smaller than 100, then no more than 2 people from
the same company may sign up for it
• Attendees from Amsterdam cannot attend sessions in room 010

• Common challenge:
– Many data change events
can lead to constraint violation
21

Event Analysis
for Inter Entity Constraint
• No more planned session attendances are allowed than the capacity of
the room in which the session is scheduled to take place
Create,
Update (session reference)

Update (room reference)

Update (capacity [decrease])
22

Constraint classification
• Based on event-analysis (when can the constraint get
violated) we discern these categories of contraints
–
–
–
–

Attribute
Tuple
Entity
Inter Entity

• Each category has its own
implementation methods,
options and considerations
– E.g. Multi record instance rules cannot
meaningfully be enforced in client/web-tier
23

Nous ne sommes
pas „Sans Famille‟
24

Nous ne sommes
pas „Sans Famille‟

Mobile Client

Client
(pure HTML 5 & Java
Script)

Client
(JSF based HTML 5 & Java Script)

Web Tier
JavaServer Faces

RESTful Services
POJO
Domain
Model

Business Tier
JPA

RDBMS

EJB
25

Multiple clients for
Data Source
Client
(pure HTML 5 & Java Script)

Mobile Client

Client
(JSF based HTML 5 & Java Script)

Web Tier
JavaServer Faces

RESTful Services

POJO Domain Model

EJB
Business Tier
JPA

Mobile Client

Client
(pure HTML 5 & Java
Script)

Client
(JSF based HTML 5 & Java Script)

Web Tier
JavaServer Faces

RESTful Services
POJO
Domain
Model

Business Tier
JPA

EJB

.NET

ESB

DBA/
Application
Admin

RDBMS

Batch
26

Integrity Enforcement in the
Persistent Store
• All data is available
• Persistent store is the final stop: the buck stops here
– Any alternative data manipulation (channel) has to go to the persistent store
– Mobile, Batch, DBA, ESB

• Built-in (native) mechanisms
for constraint enforcement
– Productive development, proven robustness, scalable performance
– For example:
Column Type, PK/UK, FK, Check; trigger

• Transactions
• Enforcing integrity is integral part of persisting data
– Without final validation, persistent store cannot take responsibility for integrity
27

Multiple-Instances-of-Single-Entity
constraints
• No more than 5 conference attendees from the same company
28

Implementation Consideration
for Multiple-Entity-Instance rule
• Implementation – how and where?
–
–
–
–
–

Is the entire set of data available
Is all associated info available
Is the data set stable?
Can the constraint elegantly be implemented (natively? good framework support?)
Are all data access paths covered?
29

Implementing Multi-Instance
constraint „5 max per company‟
Register New Attendee – method A
- Ensure L2 Cache is up to date in terms of
Attendees (fetch all attendees into cache)
- Inspect the collection of attendees for
same company
- Persist Attendee if collection does not hold
5 (or more)
POJO
Domain
Model

Register New Attendee – method B
- Select count of attendees in same
company from the Data Store
- Inspect the long value
- Persist Attendee if long is < 5

Business Tier
JPA

Attendees

L2 Cache
Attendees
30

Max 5 per company
JPA Facade enforcement
Max 5 per Company – Flaws in
JPA Enforcement
• Persist does not [always] „post to database‟
– When more than one attendee is added in a transaction, prior ones are not counted
when the latter are validated

Thread 1

POJO
Domain
Model

select count
persist
select count
persist
commit

Facade

Business Tier
JPA

Attendees

31
32

JPA Facade enforcement in a
multi-threaded world
Client
HTML 5 & Java Script
Session A

Client
HTML 5 & Java Script
Session B

Web Tier
Thread 1

POJO
Domain
Model

Thread 2

select count
persist
commit

select count
persist
commit

Facade

Business Tier
JPA

Attendees
33

Transactions
• Logically consistent set of data manipulations
– Atomic units of work
– Succeed or fail together
– Any changes inside a transaction are invisible to other sessions/transactions until the
transaction completes (commits)
– Note: during a transaction, constraints may be violated; the only thing that matters:
commit [time]
– Transaction ends with succesful commit or rollback –
In both cases, transaction-owned locks are released

• ACID (in RDBMS)
– vs BASE (in NoSQL: soft state, eventual consistency - hopefully)

• Note: post vs. commit with RDBMS
– Post means do [all] data manipulation (insert, update, delete) but do not commit [yet]
– Only upon commit are changes persisted and published
34

Perfect Integrity
35

Fine grained locking

Transaction 1
insert …
('John','Doe',…)
Lock on
UK1_JOHN_
DOE

Transaction 2
insert …
('Jane','Doe',…)

update <JANE>
set firstname ='John'
commit

Attendees

Unique Key UK1 on
(FirstName, LastName)
36

JPA Facade enforcement
Exclusive Constraint Checking
Client
HTML 5 & Java Script
Session A

Client
HTML 5 & Java Script
Session B

Web Tier
Thread 1

POJO
Domain
Model

Thread 2

take lock
select count
persist
Facade
commit

take lock…
select count
rollback

Business Tier
JPA

LockMgr
ATT_MAX

Attendees
37

Distributed or Global
Transaction
• One logical unit of work - involving data manipulations in multiple
resources (global transaction composed of local transactions)
Mobile Client

Client
(pure HTML 5 & Java
Script)

Client
(JSF based HTML 5 & Java Script)

Web Tier
JavaServer Faces

RESTful Services
POJO
Domain
Model

RDBMS

EJB
Business Tier

RDBMS

JCA

JMS

ERP
38

Implementation for
Distributed Transaction
• Typical approach: two-phase commit
– Each resource locks and validates – then reports OK or NOK back to the transaction
overseeer
– When all resources have indicated OK
then phase two:
all resources commit and
release locks
– When one or more resources signal
NOK, then phase two:
all resources roll back/undo
changes and release locks

• With regards to integrity:
– With a distributed transaction,
the integrity for each participant
is handled as before;
this will result in „constraint-locks‟ in multiple separate resources
39

Distributed (aka global)
transaction within a JVM
• Java EE containers (and various non-EE JTA implementations) support
global (distributed) transactions within a JVM
– JTA (JSR-907) – based on X/Open XA architecture

• Key element is Transaction Monitor (the container) and Resource
Managers (JDBC, EJB, JMS, JCA)
• One non-XA resource can participate (file system, email, …) in a global
transaction:
–
–
–
–

All XA-resources perform Phase One
The non-XA resource does its thing
Upon success of the non-XA resource: others perform Phase two by comitting
Upon failure of the non-XA resource: others roll back
40

Distributed transactions
across/outside containers

Step 2:
Payment

Mobile Client

Client
(pure HTML 5 & Java
Script)

Client
(JSF based HTML 5 & Java Script)

Web Tier
JavaServer Faces

RESTful Services
POJO
Domain
Model

Business Tier
JPA

RDBMS

EJB
41

Distributed transactions
across/outside containers
• Transaction involving remote containers, Web Services, File System or
any stateless transaction participant
• There is no actual common, shared vehicle (like a global XA transaction)
– There is not really a coordinated two-phase commit

• Transaction consists of
– Any resource does its thing – lock, validate, commit (or rollback), report back
– If all resources report succes: great, done
– If one resource reports failure the all other resources should perform
‘compensation’ – i.e. rollback/undo effects of a committed transaction
commit

Container
Local
Enterprise
Resource

Transaction

compensate

commit

Remote/Stateless
Enterprise
Resource
Remote/Stateless
Enterprise
Resource
42

Compensation
• How to implement a compensation mechanism?
• How long after the commit can compensation be requested?
• What is the state of the enterprise resource between commit and the
compensation expiry time?
• Should the invoker notify the resource that compensation is no longer
required (so the „logical locks‟/‟temporary state‟ can be updated)
– i.e. the global distributed transaction has succussfully completed

commit
compensate

Enterprise
Resource
RESTful “transaction” is a
distributed transaction

Client

Resource A

Resource B
Domain Model/JPA Cache

Resource C

43
RESTful “transaction” is a
distributed transaction

Client

Resource A

Resource B
Domain Model/JPA

Resource C

44
45

Distributed
Constraints
• Constraints that involve data collections in multiple enterprise resources
Mobile Client

Client
(pure HTML 5 & JS)

Client
(JSF based HTML 5 & Java Script)

Web Tier
JavaServer Faces

RESTful Services
POJO
Domain
Model

RDBMS

Table Y

Business Tier

RDBMS
Table X

EJB
JCA

JMS

ERP
46

Distributed Constraints
• Not more than three attendees (resource A) from the same company may
attend a session (resource B)
– Insert/Update Attendance requires validation – as does update of Attendee.company
Client

ESB

Client

Web Tier
Java EE
Business Tier

Client
Web Tier

MAX_3_COMP_ATT

Java EE
Business Tier

Distributed Lock
Manager
ATTENDEES

ATTENDANCES
48

Java global (distributed) lock
managers
• Within JVM: SynchronousQueue
• Across JVMs: Apache ZooKeeper, HazelCast, Oracle Coherence, …

JVM
JVM
JVM
49

Summary
• Which level of integrity is required?
• Change of data potentially undermines integrity
– Data change is trigger for constraint validation

• Exclusive lock on multi-record validation
– released when transaction commits

• Ensure that all data access paths are covered
– Not all data manipulations may come through the Java middle tier

• Transactions may include multiple enterprise resources
– That may not be able to participate in a distributed transaction and have to support a
compensation mechanism

• True integrity and real robustness are very hard to achieve
– Much harder than is commonly assumed
51

Handling Integrity Really Well...
Lucas Jellema (AMIS)

Email: lucas.jellema@amis.nl
Twitter: @lucasjellema
Blog: http://technology.amis.nl
Website: http://www.amis.nl

Weitere ähnliche Inhalte

Was ist angesagt?

Distributed DBMS - Unit 8 - Distributed Transaction Management & Concurrency ...
Distributed DBMS - Unit 8 - Distributed Transaction Management & Concurrency ...Distributed DBMS - Unit 8 - Distributed Transaction Management & Concurrency ...
Distributed DBMS - Unit 8 - Distributed Transaction Management & Concurrency ...Gyanmanjari Institute Of Technology
 
Unit iv -Transactions
Unit iv -TransactionsUnit iv -Transactions
Unit iv -TransactionsDhivyaa C.R
 
Resilience planning and how the empire strikes back
Resilience planning and how the empire strikes backResilience planning and how the empire strikes back
Resilience planning and how the empire strikes backBhakti Mehta
 
Business processes, business rules, complex event processing, the JBoss way
Business processes, business rules, complex event processing, the JBoss wayBusiness processes, business rules, complex event processing, the JBoss way
Business processes, business rules, complex event processing, the JBoss wayKris Verlaenen
 
Streamline your processes with jBPM 6
Streamline your processes with jBPM 6Streamline your processes with jBPM 6
Streamline your processes with jBPM 6jsvitak
 

Was ist angesagt? (6)

Distributed DBMS - Unit 8 - Distributed Transaction Management & Concurrency ...
Distributed DBMS - Unit 8 - Distributed Transaction Management & Concurrency ...Distributed DBMS - Unit 8 - Distributed Transaction Management & Concurrency ...
Distributed DBMS - Unit 8 - Distributed Transaction Management & Concurrency ...
 
Unit iv -Transactions
Unit iv -TransactionsUnit iv -Transactions
Unit iv -Transactions
 
Resilience planning and how the empire strikes back
Resilience planning and how the empire strikes backResilience planning and how the empire strikes back
Resilience planning and how the empire strikes back
 
An overview of BizTalk
An overview of BizTalkAn overview of BizTalk
An overview of BizTalk
 
Business processes, business rules, complex event processing, the JBoss way
Business processes, business rules, complex event processing, the JBoss wayBusiness processes, business rules, complex event processing, the JBoss way
Business processes, business rules, complex event processing, the JBoss way
 
Streamline your processes with jBPM 6
Streamline your processes with jBPM 6Streamline your processes with jBPM 6
Streamline your processes with jBPM 6
 

Andere mochten auch

Ing presentatie j fall 2013 nijkerk amir arooni
Ing presentatie j fall 2013 nijkerk amir arooniIng presentatie j fall 2013 nijkerk amir arooni
Ing presentatie j fall 2013 nijkerk amir arooniNLJUG
 
How to get more kids to code
How to get more kids to codeHow to get more kids to code
How to get more kids to codeNLJUG
 
Web-scale op basis van Hadoop en Akka Reactive Streams - Johan Tillema, Rene ...
Web-scale op basis van Hadoop en Akka Reactive Streams - Johan Tillema, Rene ...Web-scale op basis van Hadoop en Akka Reactive Streams - Johan Tillema, Rene ...
Web-scale op basis van Hadoop en Akka Reactive Streams - Johan Tillema, Rene ...NLJUG
 
Workshop spring boot presentatie - Atos
Workshop spring boot presentatie - AtosWorkshop spring boot presentatie - Atos
Workshop spring boot presentatie - AtosNLJUG
 
Cultivating the jenkins job jungle with groovy - Patrick van Dissel
Cultivating the jenkins job jungle with groovy - Patrick van DisselCultivating the jenkins job jungle with groovy - Patrick van Dissel
Cultivating the jenkins job jungle with groovy - Patrick van DisselNLJUG
 
J fall 2013 - Hands-on lab 'effectief automatisch testen met cucumber'
J fall 2013 - Hands-on lab 'effectief automatisch testen met cucumber'J fall 2013 - Hands-on lab 'effectief automatisch testen met cucumber'
J fall 2013 - Hands-on lab 'effectief automatisch testen met cucumber'NLJUG
 
Kill the mutants and test your tests - Roy van Rijn
Kill the mutants and test your tests - Roy van RijnKill the mutants and test your tests - Roy van Rijn
Kill the mutants and test your tests - Roy van RijnNLJUG
 
Reactieve applicaties allard buijze
Reactieve applicaties   allard buijzeReactieve applicaties   allard buijze
Reactieve applicaties allard buijzeNLJUG
 
Decoding the airspace above you with Java and $7 hardware - Bert Jan Schrijver
Decoding the airspace above you with Java and $7 hardware - Bert Jan SchrijverDecoding the airspace above you with Java and $7 hardware - Bert Jan Schrijver
Decoding the airspace above you with Java and $7 hardware - Bert Jan SchrijverNLJUG
 
Speedy perception trumps speedy reception–smart asynchronous interactions - L...
Speedy perception trumps speedy reception–smart asynchronous interactions - L...Speedy perception trumps speedy reception–smart asynchronous interactions - L...
Speedy perception trumps speedy reception–smart asynchronous interactions - L...NLJUG
 
Scratching the Surface with JavaFX
Scratching the Surface with JavaFXScratching the Surface with JavaFX
Scratching the Surface with JavaFXNLJUG
 
Using Docker to Develop, Test and Run Maven Projects - Wouter Danes
Using Docker to Develop, Test and Run Maven Projects - Wouter DanesUsing Docker to Develop, Test and Run Maven Projects - Wouter Danes
Using Docker to Develop, Test and Run Maven Projects - Wouter DanesNLJUG
 
Gebruik dezelfde Docker container voor Java applicaties tijdens ontwikkelen e...
Gebruik dezelfde Docker container voor Java applicaties tijdens ontwikkelen e...Gebruik dezelfde Docker container voor Java applicaties tijdens ontwikkelen e...
Gebruik dezelfde Docker container voor Java applicaties tijdens ontwikkelen e...NLJUG
 
Real-World Scala Design Patterns
Real-World Scala Design PatternsReal-World Scala Design Patterns
Real-World Scala Design PatternsNLJUG
 
Shootout! template engines on the jvm
Shootout! template engines on the jvmShootout! template engines on the jvm
Shootout! template engines on the jvmNLJUG
 
Opening J-Fall 2013
Opening J-Fall 2013Opening J-Fall 2013
Opening J-Fall 2013NLJUG
 
Akka in Practice: Designing Actor-based Applications
Akka in Practice: Designing Actor-based ApplicationsAkka in Practice: Designing Actor-based Applications
Akka in Practice: Designing Actor-based ApplicationsNLJUG
 

Andere mochten auch (17)

Ing presentatie j fall 2013 nijkerk amir arooni
Ing presentatie j fall 2013 nijkerk amir arooniIng presentatie j fall 2013 nijkerk amir arooni
Ing presentatie j fall 2013 nijkerk amir arooni
 
How to get more kids to code
How to get more kids to codeHow to get more kids to code
How to get more kids to code
 
Web-scale op basis van Hadoop en Akka Reactive Streams - Johan Tillema, Rene ...
Web-scale op basis van Hadoop en Akka Reactive Streams - Johan Tillema, Rene ...Web-scale op basis van Hadoop en Akka Reactive Streams - Johan Tillema, Rene ...
Web-scale op basis van Hadoop en Akka Reactive Streams - Johan Tillema, Rene ...
 
Workshop spring boot presentatie - Atos
Workshop spring boot presentatie - AtosWorkshop spring boot presentatie - Atos
Workshop spring boot presentatie - Atos
 
Cultivating the jenkins job jungle with groovy - Patrick van Dissel
Cultivating the jenkins job jungle with groovy - Patrick van DisselCultivating the jenkins job jungle with groovy - Patrick van Dissel
Cultivating the jenkins job jungle with groovy - Patrick van Dissel
 
J fall 2013 - Hands-on lab 'effectief automatisch testen met cucumber'
J fall 2013 - Hands-on lab 'effectief automatisch testen met cucumber'J fall 2013 - Hands-on lab 'effectief automatisch testen met cucumber'
J fall 2013 - Hands-on lab 'effectief automatisch testen met cucumber'
 
Kill the mutants and test your tests - Roy van Rijn
Kill the mutants and test your tests - Roy van RijnKill the mutants and test your tests - Roy van Rijn
Kill the mutants and test your tests - Roy van Rijn
 
Reactieve applicaties allard buijze
Reactieve applicaties   allard buijzeReactieve applicaties   allard buijze
Reactieve applicaties allard buijze
 
Decoding the airspace above you with Java and $7 hardware - Bert Jan Schrijver
Decoding the airspace above you with Java and $7 hardware - Bert Jan SchrijverDecoding the airspace above you with Java and $7 hardware - Bert Jan Schrijver
Decoding the airspace above you with Java and $7 hardware - Bert Jan Schrijver
 
Speedy perception trumps speedy reception–smart asynchronous interactions - L...
Speedy perception trumps speedy reception–smart asynchronous interactions - L...Speedy perception trumps speedy reception–smart asynchronous interactions - L...
Speedy perception trumps speedy reception–smart asynchronous interactions - L...
 
Scratching the Surface with JavaFX
Scratching the Surface with JavaFXScratching the Surface with JavaFX
Scratching the Surface with JavaFX
 
Using Docker to Develop, Test and Run Maven Projects - Wouter Danes
Using Docker to Develop, Test and Run Maven Projects - Wouter DanesUsing Docker to Develop, Test and Run Maven Projects - Wouter Danes
Using Docker to Develop, Test and Run Maven Projects - Wouter Danes
 
Gebruik dezelfde Docker container voor Java applicaties tijdens ontwikkelen e...
Gebruik dezelfde Docker container voor Java applicaties tijdens ontwikkelen e...Gebruik dezelfde Docker container voor Java applicaties tijdens ontwikkelen e...
Gebruik dezelfde Docker container voor Java applicaties tijdens ontwikkelen e...
 
Real-World Scala Design Patterns
Real-World Scala Design PatternsReal-World Scala Design Patterns
Real-World Scala Design Patterns
 
Shootout! template engines on the jvm
Shootout! template engines on the jvmShootout! template engines on the jvm
Shootout! template engines on the jvm
 
Opening J-Fall 2013
Opening J-Fall 2013Opening J-Fall 2013
Opening J-Fall 2013
 
Akka in Practice: Designing Actor-based Applications
Akka in Practice: Designing Actor-based ApplicationsAkka in Practice: Designing Actor-based Applications
Akka in Practice: Designing Actor-based Applications
 

Ähnlich wie On the integrity of data in Java Applications

Data Integrity in Java Applications (JFall 2013)
Data Integrity in Java Applications (JFall 2013)Data Integrity in Java Applications (JFall 2013)
Data Integrity in Java Applications (JFall 2013)Lucas Jellema
 
Scaling Systems: Architectures that grow
Scaling Systems: Architectures that growScaling Systems: Architectures that grow
Scaling Systems: Architectures that growGibraltar Software
 
Impact 2008 1994A - Exposing services people want to consume: a model-driven ...
Impact 2008 1994A - Exposing services people want to consume: a model-driven ...Impact 2008 1994A - Exposing services people want to consume: a model-driven ...
Impact 2008 1994A - Exposing services people want to consume: a model-driven ...Brian Petrini
 
CQRS + Event Sourcing
CQRS + Event SourcingCQRS + Event Sourcing
CQRS + Event SourcingMike Bild
 
Resilience Planning & How the Empire Strikes Back
Resilience Planning & How the Empire Strikes BackResilience Planning & How the Empire Strikes Back
Resilience Planning & How the Empire Strikes BackC4Media
 
Why retail companies can't afford database downtime
Why retail companies can't afford database downtimeWhy retail companies can't afford database downtime
Why retail companies can't afford database downtimeDBmaestro - Database DevOps
 
Transaction management and concurrency control
Transaction management and concurrency controlTransaction management and concurrency control
Transaction management and concurrency controlDhani Ahmad
 
Architectural Decisions: Smoothly and Consistently
Architectural Decisions: Smoothly and ConsistentlyArchitectural Decisions: Smoothly and Consistently
Architectural Decisions: Smoothly and ConsistentlyComsysto Reply GmbH
 
Architectural Decisions: Smoothly and Consistently
Architectural Decisions: Smoothly and ConsistentlyArchitectural Decisions: Smoothly and Consistently
Architectural Decisions: Smoothly and ConsistentlyComsysto Reply GmbH
 
Adding Value in the Cloud with Performance Test
Adding Value in the Cloud with Performance TestAdding Value in the Cloud with Performance Test
Adding Value in the Cloud with Performance TestRodolfo Kohn
 
Dynamics CRM high volume systems - lessons from the field
Dynamics CRM high volume systems - lessons from the fieldDynamics CRM high volume systems - lessons from the field
Dynamics CRM high volume systems - lessons from the fieldStéphane Dorrekens
 
AAI-2013 Preparing to Fail: Practical WebSphere Application Server High Avail...
AAI-2013 Preparing to Fail: Practical WebSphere Application Server High Avail...AAI-2013 Preparing to Fail: Practical WebSphere Application Server High Avail...
AAI-2013 Preparing to Fail: Practical WebSphere Application Server High Avail...WASdev Community
 
Software Architecture and Architectors: useless VS valuable
Software Architecture and Architectors: useless VS valuableSoftware Architecture and Architectors: useless VS valuable
Software Architecture and Architectors: useless VS valuableComsysto Reply GmbH
 
Foundational Design Patterns for Multi-Purpose Applications
Foundational Design Patterns for Multi-Purpose ApplicationsFoundational Design Patterns for Multi-Purpose Applications
Foundational Design Patterns for Multi-Purpose ApplicationsChing-Hwa Yu
 
VMworld Europe 2014: Virtualizing Databases Doing IT Right – The Sequel
VMworld Europe 2014: Virtualizing Databases Doing IT Right – The SequelVMworld Europe 2014: Virtualizing Databases Doing IT Right – The Sequel
VMworld Europe 2014: Virtualizing Databases Doing IT Right – The SequelVMworld
 
BEA_eworld_2001_jta.ppt
BEA_eworld_2001_jta.pptBEA_eworld_2001_jta.ppt
BEA_eworld_2001_jta.pptssuser670564
 
Микола Ковш “Performance Testing Implementation From Scratch. Why? When and H...
Микола Ковш “Performance Testing Implementation From Scratch. Why? When and H...Микола Ковш “Performance Testing Implementation From Scratch. Why? When and H...
Микола Ковш “Performance Testing Implementation From Scratch. Why? When and H...Dakiry
 

Ähnlich wie On the integrity of data in Java Applications (20)

Data Integrity in Java Applications (JFall 2013)
Data Integrity in Java Applications (JFall 2013)Data Integrity in Java Applications (JFall 2013)
Data Integrity in Java Applications (JFall 2013)
 
Scaling Systems: Architectures that grow
Scaling Systems: Architectures that growScaling Systems: Architectures that grow
Scaling Systems: Architectures that grow
 
Impact 2008 1994A - Exposing services people want to consume: a model-driven ...
Impact 2008 1994A - Exposing services people want to consume: a model-driven ...Impact 2008 1994A - Exposing services people want to consume: a model-driven ...
Impact 2008 1994A - Exposing services people want to consume: a model-driven ...
 
CQRS + Event Sourcing
CQRS + Event SourcingCQRS + Event Sourcing
CQRS + Event Sourcing
 
Resilience Planning & How the Empire Strikes Back
Resilience Planning & How the Empire Strikes BackResilience Planning & How the Empire Strikes Back
Resilience Planning & How the Empire Strikes Back
 
Why retail companies can't afford database downtime
Why retail companies can't afford database downtimeWhy retail companies can't afford database downtime
Why retail companies can't afford database downtime
 
Transaction management and concurrency control
Transaction management and concurrency controlTransaction management and concurrency control
Transaction management and concurrency control
 
Hbase hive pig
Hbase hive pigHbase hive pig
Hbase hive pig
 
Architectural Decisions: Smoothly and Consistently
Architectural Decisions: Smoothly and ConsistentlyArchitectural Decisions: Smoothly and Consistently
Architectural Decisions: Smoothly and Consistently
 
Architectural Decisions: Smoothly and Consistently
Architectural Decisions: Smoothly and ConsistentlyArchitectural Decisions: Smoothly and Consistently
Architectural Decisions: Smoothly and Consistently
 
Adding Value in the Cloud with Performance Test
Adding Value in the Cloud with Performance TestAdding Value in the Cloud with Performance Test
Adding Value in the Cloud with Performance Test
 
Dynamics CRM high volume systems - lessons from the field
Dynamics CRM high volume systems - lessons from the fieldDynamics CRM high volume systems - lessons from the field
Dynamics CRM high volume systems - lessons from the field
 
AAI-2013 Preparing to Fail: Practical WebSphere Application Server High Avail...
AAI-2013 Preparing to Fail: Practical WebSphere Application Server High Avail...AAI-2013 Preparing to Fail: Practical WebSphere Application Server High Avail...
AAI-2013 Preparing to Fail: Practical WebSphere Application Server High Avail...
 
Software Architecture and Architectors: useless VS valuable
Software Architecture and Architectors: useless VS valuableSoftware Architecture and Architectors: useless VS valuable
Software Architecture and Architectors: useless VS valuable
 
Foundational Design Patterns for Multi-Purpose Applications
Foundational Design Patterns for Multi-Purpose ApplicationsFoundational Design Patterns for Multi-Purpose Applications
Foundational Design Patterns for Multi-Purpose Applications
 
ADF Performance Monitor
ADF Performance MonitorADF Performance Monitor
ADF Performance Monitor
 
VMworld Europe 2014: Virtualizing Databases Doing IT Right – The Sequel
VMworld Europe 2014: Virtualizing Databases Doing IT Right – The SequelVMworld Europe 2014: Virtualizing Databases Doing IT Right – The Sequel
VMworld Europe 2014: Virtualizing Databases Doing IT Right – The Sequel
 
BEA_eworld_2001_jta.ppt
BEA_eworld_2001_jta.pptBEA_eworld_2001_jta.ppt
BEA_eworld_2001_jta.ppt
 
Микола Ковш “Performance Testing Implementation From Scratch. Why? When and H...
Микола Ковш “Performance Testing Implementation From Scratch. Why? When and H...Микола Ковш “Performance Testing Implementation From Scratch. Why? When and H...
Микола Ковш “Performance Testing Implementation From Scratch. Why? When and H...
 
JBUG.be jBPM4
JBUG.be jBPM4JBUG.be jBPM4
JBUG.be jBPM4
 

Mehr von NLJUG

The future of Web-Scale - Johan Tillema, Rene Boere & Chris Quach
The future of Web-Scale - Johan Tillema, Rene Boere & Chris QuachThe future of Web-Scale - Johan Tillema, Rene Boere & Chris Quach
The future of Web-Scale - Johan Tillema, Rene Boere & Chris QuachNLJUG
 
Real-time user interfaces - sosm gewoon makkelijker - Allard Buijze
Real-time user interfaces - sosm gewoon makkelijker - Allard BuijzeReal-time user interfaces - sosm gewoon makkelijker - Allard Buijze
Real-time user interfaces - sosm gewoon makkelijker - Allard BuijzeNLJUG
 
The end of traditional enterprise IT - ING's journey to the next generation I...
The end of traditional enterprise IT - ING's journey to the next generation I...The end of traditional enterprise IT - ING's journey to the next generation I...
The end of traditional enterprise IT - ING's journey to the next generation I...NLJUG
 
Performance van Java 8 en verder - Jeroen Borgers
Performance van Java 8 en verder - Jeroen BorgersPerformance van Java 8 en verder - Jeroen Borgers
Performance van Java 8 en verder - Jeroen BorgersNLJUG
 
Introduction to Reactive with Play and Akka - Markus Jura
Introduction to Reactive with Play and Akka - Markus JuraIntroduction to Reactive with Play and Akka - Markus Jura
Introduction to Reactive with Play and Akka - Markus JuraNLJUG
 
Workshop angular dart presentatie - Atos
Workshop angular dart presentatie - AtosWorkshop angular dart presentatie - Atos
Workshop angular dart presentatie - AtosNLJUG
 
Rethink your architecture - Marten Deinum
Rethink your architecture - Marten DeinumRethink your architecture - Marten Deinum
Rethink your architecture - Marten DeinumNLJUG
 
Evolutionary Algorithms: the key to solving complex Java puzzles! - Bas knopper
Evolutionary Algorithms: the key to solving complex Java puzzles! - Bas knopperEvolutionary Algorithms: the key to solving complex Java puzzles! - Bas knopper
Evolutionary Algorithms: the key to solving complex Java puzzles! - Bas knopperNLJUG
 
Modularity and Domain Driven Design; a killer Combination? - Tom de Wolf & St...
Modularity and Domain Driven Design; a killer Combination? - Tom de Wolf & St...Modularity and Domain Driven Design; a killer Combination? - Tom de Wolf & St...
Modularity and Domain Driven Design; a killer Combination? - Tom de Wolf & St...NLJUG
 
Apache Wicket: 10 jaar en verder - Martijn Dashorst
Apache Wicket: 10 jaar en verder - Martijn DashorstApache Wicket: 10 jaar en verder - Martijn Dashorst
Apache Wicket: 10 jaar en verder - Martijn DashorstNLJUG
 
Opening - Bert Ertman
Opening - Bert ErtmanOpening - Bert Ertman
Opening - Bert ErtmanNLJUG
 
Returning the right results - Jettro Coenradie
Returning the right results - Jettro CoenradieReturning the right results - Jettro Coenradie
Returning the right results - Jettro CoenradieNLJUG
 
Reactive programming met Java 8 en Java EE 7 - Martijn Blankestijn
Reactive programming met Java 8 en Java EE 7 - Martijn BlankestijnReactive programming met Java 8 en Java EE 7 - Martijn Blankestijn
Reactive programming met Java 8 en Java EE 7 - Martijn BlankestijnNLJUG
 
Event-sourced architectures with Akka - Sander Mak
Event-sourced architectures with Akka - Sander MakEvent-sourced architectures with Akka - Sander Mak
Event-sourced architectures with Akka - Sander MakNLJUG
 
Setting up a mini big data architecture, just for you! - Bas Geerdink
Setting up a mini big data architecture, just for you! - Bas GeerdinkSetting up a mini big data architecture, just for you! - Bas Geerdink
Setting up a mini big data architecture, just for you! - Bas GeerdinkNLJUG
 
A Storm of Drones - Dennis Vredeveld
A Storm of Drones - Dennis Vredeveld A Storm of Drones - Dennis Vredeveld
A Storm of Drones - Dennis Vredeveld NLJUG
 
ING : How top quality software and state-of-the-art technology leads to conti...
ING : How top quality software and state-of-the-art technology leads to conti...ING : How top quality software and state-of-the-art technology leads to conti...
ING : How top quality software and state-of-the-art technology leads to conti...NLJUG
 
Data Science with R for Java Developers
Data Science with R for Java DevelopersData Science with R for Java Developers
Data Science with R for Java DevelopersNLJUG
 
Modular JavaScript
Modular JavaScriptModular JavaScript
Modular JavaScriptNLJUG
 
JVM bytecode engineering 101
JVM bytecode engineering 101JVM bytecode engineering 101
JVM bytecode engineering 101NLJUG
 

Mehr von NLJUG (20)

The future of Web-Scale - Johan Tillema, Rene Boere & Chris Quach
The future of Web-Scale - Johan Tillema, Rene Boere & Chris QuachThe future of Web-Scale - Johan Tillema, Rene Boere & Chris Quach
The future of Web-Scale - Johan Tillema, Rene Boere & Chris Quach
 
Real-time user interfaces - sosm gewoon makkelijker - Allard Buijze
Real-time user interfaces - sosm gewoon makkelijker - Allard BuijzeReal-time user interfaces - sosm gewoon makkelijker - Allard Buijze
Real-time user interfaces - sosm gewoon makkelijker - Allard Buijze
 
The end of traditional enterprise IT - ING's journey to the next generation I...
The end of traditional enterprise IT - ING's journey to the next generation I...The end of traditional enterprise IT - ING's journey to the next generation I...
The end of traditional enterprise IT - ING's journey to the next generation I...
 
Performance van Java 8 en verder - Jeroen Borgers
Performance van Java 8 en verder - Jeroen BorgersPerformance van Java 8 en verder - Jeroen Borgers
Performance van Java 8 en verder - Jeroen Borgers
 
Introduction to Reactive with Play and Akka - Markus Jura
Introduction to Reactive with Play and Akka - Markus JuraIntroduction to Reactive with Play and Akka - Markus Jura
Introduction to Reactive with Play and Akka - Markus Jura
 
Workshop angular dart presentatie - Atos
Workshop angular dart presentatie - AtosWorkshop angular dart presentatie - Atos
Workshop angular dart presentatie - Atos
 
Rethink your architecture - Marten Deinum
Rethink your architecture - Marten DeinumRethink your architecture - Marten Deinum
Rethink your architecture - Marten Deinum
 
Evolutionary Algorithms: the key to solving complex Java puzzles! - Bas knopper
Evolutionary Algorithms: the key to solving complex Java puzzles! - Bas knopperEvolutionary Algorithms: the key to solving complex Java puzzles! - Bas knopper
Evolutionary Algorithms: the key to solving complex Java puzzles! - Bas knopper
 
Modularity and Domain Driven Design; a killer Combination? - Tom de Wolf & St...
Modularity and Domain Driven Design; a killer Combination? - Tom de Wolf & St...Modularity and Domain Driven Design; a killer Combination? - Tom de Wolf & St...
Modularity and Domain Driven Design; a killer Combination? - Tom de Wolf & St...
 
Apache Wicket: 10 jaar en verder - Martijn Dashorst
Apache Wicket: 10 jaar en verder - Martijn DashorstApache Wicket: 10 jaar en verder - Martijn Dashorst
Apache Wicket: 10 jaar en verder - Martijn Dashorst
 
Opening - Bert Ertman
Opening - Bert ErtmanOpening - Bert Ertman
Opening - Bert Ertman
 
Returning the right results - Jettro Coenradie
Returning the right results - Jettro CoenradieReturning the right results - Jettro Coenradie
Returning the right results - Jettro Coenradie
 
Reactive programming met Java 8 en Java EE 7 - Martijn Blankestijn
Reactive programming met Java 8 en Java EE 7 - Martijn BlankestijnReactive programming met Java 8 en Java EE 7 - Martijn Blankestijn
Reactive programming met Java 8 en Java EE 7 - Martijn Blankestijn
 
Event-sourced architectures with Akka - Sander Mak
Event-sourced architectures with Akka - Sander MakEvent-sourced architectures with Akka - Sander Mak
Event-sourced architectures with Akka - Sander Mak
 
Setting up a mini big data architecture, just for you! - Bas Geerdink
Setting up a mini big data architecture, just for you! - Bas GeerdinkSetting up a mini big data architecture, just for you! - Bas Geerdink
Setting up a mini big data architecture, just for you! - Bas Geerdink
 
A Storm of Drones - Dennis Vredeveld
A Storm of Drones - Dennis Vredeveld A Storm of Drones - Dennis Vredeveld
A Storm of Drones - Dennis Vredeveld
 
ING : How top quality software and state-of-the-art technology leads to conti...
ING : How top quality software and state-of-the-art technology leads to conti...ING : How top quality software and state-of-the-art technology leads to conti...
ING : How top quality software and state-of-the-art technology leads to conti...
 
Data Science with R for Java Developers
Data Science with R for Java DevelopersData Science with R for Java Developers
Data Science with R for Java Developers
 
Modular JavaScript
Modular JavaScriptModular JavaScript
Modular JavaScript
 
JVM bytecode engineering 101
JVM bytecode engineering 101JVM bytecode engineering 101
JVM bytecode engineering 101
 

On the integrity of data in Java Applications

  • 1. On the integrity of data in Java Applications Lucas Jellema (AMIS) NLJUG JFall 2013 6th November 2013, Nijkerk, The Netherlands
  • 2. Agenda • What is integrity? • Enforcing data constraints – throughout the application architecture • Transactions • Exclusive Access to … • The Distributed World
  • 3. 3 Definition of Integrity • Truth – Nothing but the truth • The Only Truth • [Degree of] success or completeness of actions is known
  • 6. 6 Conference Application Client (HTML 5 & Java Script) Web Tier JavaServer Faces POJO Domain Model Business Tier JPA RDBMS EJB
  • 8. 8 Validation at entry time Client and View
  • 9. 9 Validation at entry time Client and View
  • 10. More validation at entry time – bean Validation 10
  • 11. 11 Validation at entry time Bean Validation in View
  • 12. 13 Record (Type) level rules • Program should be Kids when age < 18; and Management or Developer when age > 18 • Using JavaScript – when either field changes (handle nulls) – on submit of the entire record • Using Bean Validation: custom type validator – in either web-tier or JPA
  • 13. 14 Type Level Constraints with Bean Validation
  • 14. 15 Type Level Bean Validation: Custom Validator
  • 15. 16 Validation Implementation options & considerations Native Mobile Client Native HTML 5; JavaScript Client (pure HTML 5 & Java Script) Native HTML 5; JavaScript Client (JSF based HTML 5 & Java Script) Custom; Web Tier JSF Validator; Bean JavaServer Faces Validation Custom; Bean Validation RESTful Services POJO Domain Model Business Tier JPA RDBMS EJB Custom; Bean Validation
  • 16. 17 But wait – there is more! • More User Interfaces • More Attendee • • • • Instances More Entities & More types of Constraints More Users, Sessions, and Transactions More Nodes in the Middle Tier Cluster More Data Stores
  • 18. 19 Multiple-Instances-of-Single-Entity constraints • Constraints that cover multiple same type objects/instances – – – – – – Attendee‟s Registration Id is unique No more than 5 conference attendees from the same company Not more than two sessions by the same speaker At most one session scheduled per room per slot Only one keynote session in a slot Sessions from up to a maximum of three tracks can be scheduled in the same room
  • 19. 20 Inter entity constraints • Attendees can only attend one hands-on session during the conference • A person cannot attend another session in a slot in which the session (s)he is speaker of is scheduled • No more planned session attendances are allowed than the capacity of the room in which the session is scheduled to take place • If the room capacity is smaller than 100, then no more than 2 people from the same company may sign up for it • Attendees from Amsterdam cannot attend sessions in room 010 • Common challenge: – Many data change events can lead to constraint violation
  • 20. 21 Event Analysis for Inter Entity Constraint • No more planned session attendances are allowed than the capacity of the room in which the session is scheduled to take place Create, Update (session reference) Update (room reference) Update (capacity [decrease])
  • 21. 22 Constraint classification • Based on event-analysis (when can the constraint get violated) we discern these categories of contraints – – – – Attribute Tuple Entity Inter Entity • Each category has its own implementation methods, options and considerations – E.g. Multi record instance rules cannot meaningfully be enforced in client/web-tier
  • 22. 23 Nous ne sommes pas „Sans Famille‟
  • 23. 24 Nous ne sommes pas „Sans Famille‟ Mobile Client Client (pure HTML 5 & Java Script) Client (JSF based HTML 5 & Java Script) Web Tier JavaServer Faces RESTful Services POJO Domain Model Business Tier JPA RDBMS EJB
  • 24. 25 Multiple clients for Data Source Client (pure HTML 5 & Java Script) Mobile Client Client (JSF based HTML 5 & Java Script) Web Tier JavaServer Faces RESTful Services POJO Domain Model EJB Business Tier JPA Mobile Client Client (pure HTML 5 & Java Script) Client (JSF based HTML 5 & Java Script) Web Tier JavaServer Faces RESTful Services POJO Domain Model Business Tier JPA EJB .NET ESB DBA/ Application Admin RDBMS Batch
  • 25. 26 Integrity Enforcement in the Persistent Store • All data is available • Persistent store is the final stop: the buck stops here – Any alternative data manipulation (channel) has to go to the persistent store – Mobile, Batch, DBA, ESB • Built-in (native) mechanisms for constraint enforcement – Productive development, proven robustness, scalable performance – For example: Column Type, PK/UK, FK, Check; trigger • Transactions • Enforcing integrity is integral part of persisting data – Without final validation, persistent store cannot take responsibility for integrity
  • 26. 27 Multiple-Instances-of-Single-Entity constraints • No more than 5 conference attendees from the same company
  • 27. 28 Implementation Consideration for Multiple-Entity-Instance rule • Implementation – how and where? – – – – – Is the entire set of data available Is all associated info available Is the data set stable? Can the constraint elegantly be implemented (natively? good framework support?) Are all data access paths covered?
  • 28. 29 Implementing Multi-Instance constraint „5 max per company‟ Register New Attendee – method A - Ensure L2 Cache is up to date in terms of Attendees (fetch all attendees into cache) - Inspect the collection of attendees for same company - Persist Attendee if collection does not hold 5 (or more) POJO Domain Model Register New Attendee – method B - Select count of attendees in same company from the Data Store - Inspect the long value - Persist Attendee if long is < 5 Business Tier JPA Attendees L2 Cache Attendees
  • 29. 30 Max 5 per company JPA Facade enforcement
  • 30. Max 5 per Company – Flaws in JPA Enforcement • Persist does not [always] „post to database‟ – When more than one attendee is added in a transaction, prior ones are not counted when the latter are validated Thread 1 POJO Domain Model select count persist select count persist commit Facade Business Tier JPA Attendees 31
  • 31. 32 JPA Facade enforcement in a multi-threaded world Client HTML 5 & Java Script Session A Client HTML 5 & Java Script Session B Web Tier Thread 1 POJO Domain Model Thread 2 select count persist commit select count persist commit Facade Business Tier JPA Attendees
  • 32. 33 Transactions • Logically consistent set of data manipulations – Atomic units of work – Succeed or fail together – Any changes inside a transaction are invisible to other sessions/transactions until the transaction completes (commits) – Note: during a transaction, constraints may be violated; the only thing that matters: commit [time] – Transaction ends with succesful commit or rollback – In both cases, transaction-owned locks are released • ACID (in RDBMS) – vs BASE (in NoSQL: soft state, eventual consistency - hopefully) • Note: post vs. commit with RDBMS – Post means do [all] data manipulation (insert, update, delete) but do not commit [yet] – Only upon commit are changes persisted and published
  • 34. 35 Fine grained locking Transaction 1 insert … ('John','Doe',…) Lock on UK1_JOHN_ DOE Transaction 2 insert … ('Jane','Doe',…) update <JANE> set firstname ='John' commit Attendees Unique Key UK1 on (FirstName, LastName)
  • 35. 36 JPA Facade enforcement Exclusive Constraint Checking Client HTML 5 & Java Script Session A Client HTML 5 & Java Script Session B Web Tier Thread 1 POJO Domain Model Thread 2 take lock select count persist Facade commit take lock… select count rollback Business Tier JPA LockMgr ATT_MAX Attendees
  • 36. 37 Distributed or Global Transaction • One logical unit of work - involving data manipulations in multiple resources (global transaction composed of local transactions) Mobile Client Client (pure HTML 5 & Java Script) Client (JSF based HTML 5 & Java Script) Web Tier JavaServer Faces RESTful Services POJO Domain Model RDBMS EJB Business Tier RDBMS JCA JMS ERP
  • 37. 38 Implementation for Distributed Transaction • Typical approach: two-phase commit – Each resource locks and validates – then reports OK or NOK back to the transaction overseeer – When all resources have indicated OK then phase two: all resources commit and release locks – When one or more resources signal NOK, then phase two: all resources roll back/undo changes and release locks • With regards to integrity: – With a distributed transaction, the integrity for each participant is handled as before; this will result in „constraint-locks‟ in multiple separate resources
  • 38. 39 Distributed (aka global) transaction within a JVM • Java EE containers (and various non-EE JTA implementations) support global (distributed) transactions within a JVM – JTA (JSR-907) – based on X/Open XA architecture • Key element is Transaction Monitor (the container) and Resource Managers (JDBC, EJB, JMS, JCA) • One non-XA resource can participate (file system, email, …) in a global transaction: – – – – All XA-resources perform Phase One The non-XA resource does its thing Upon success of the non-XA resource: others perform Phase two by comitting Upon failure of the non-XA resource: others roll back
  • 39. 40 Distributed transactions across/outside containers Step 2: Payment Mobile Client Client (pure HTML 5 & Java Script) Client (JSF based HTML 5 & Java Script) Web Tier JavaServer Faces RESTful Services POJO Domain Model Business Tier JPA RDBMS EJB
  • 40. 41 Distributed transactions across/outside containers • Transaction involving remote containers, Web Services, File System or any stateless transaction participant • There is no actual common, shared vehicle (like a global XA transaction) – There is not really a coordinated two-phase commit • Transaction consists of – Any resource does its thing – lock, validate, commit (or rollback), report back – If all resources report succes: great, done – If one resource reports failure the all other resources should perform ‘compensation’ – i.e. rollback/undo effects of a committed transaction commit Container Local Enterprise Resource Transaction compensate commit Remote/Stateless Enterprise Resource Remote/Stateless Enterprise Resource
  • 41. 42 Compensation • How to implement a compensation mechanism? • How long after the commit can compensation be requested? • What is the state of the enterprise resource between commit and the compensation expiry time? • Should the invoker notify the resource that compensation is no longer required (so the „logical locks‟/‟temporary state‟ can be updated) – i.e. the global distributed transaction has succussfully completed commit compensate Enterprise Resource
  • 42. RESTful “transaction” is a distributed transaction Client Resource A Resource B Domain Model/JPA Cache Resource C 43
  • 43. RESTful “transaction” is a distributed transaction Client Resource A Resource B Domain Model/JPA Resource C 44
  • 44. 45 Distributed Constraints • Constraints that involve data collections in multiple enterprise resources Mobile Client Client (pure HTML 5 & JS) Client (JSF based HTML 5 & Java Script) Web Tier JavaServer Faces RESTful Services POJO Domain Model RDBMS Table Y Business Tier RDBMS Table X EJB JCA JMS ERP
  • 45. 46 Distributed Constraints • Not more than three attendees (resource A) from the same company may attend a session (resource B) – Insert/Update Attendance requires validation – as does update of Attendee.company Client ESB Client Web Tier Java EE Business Tier Client Web Tier MAX_3_COMP_ATT Java EE Business Tier Distributed Lock Manager ATTENDEES ATTENDANCES
  • 46. 48 Java global (distributed) lock managers • Within JVM: SynchronousQueue • Across JVMs: Apache ZooKeeper, HazelCast, Oracle Coherence, … JVM JVM JVM
  • 47. 49 Summary • Which level of integrity is required? • Change of data potentially undermines integrity – Data change is trigger for constraint validation • Exclusive lock on multi-record validation – released when transaction commits • Ensure that all data access paths are covered – Not all data manipulations may come through the Java middle tier • Transactions may include multiple enterprise resources – That may not be able to participate in a distributed transaction and have to support a compensation mechanism • True integrity and real robustness are very hard to achieve – Much harder than is commonly assumed
  • 49. Lucas Jellema (AMIS) Email: lucas.jellema@amis.nl Twitter: @lucasjellema Blog: http://technology.amis.nl Website: http://www.amis.nl

Hinweis der Redaktion

  1. Alternative UI on top of same data serviceRequestmanipulationIncreasingly Complex constraints=&gt; consider (also) server side validationUsing BeanValidation (1.0 JSR 303 Java EE 6, 1.1 JSR 349 Java EE 8?)Validationhooks in frameworkCustom code
  2. Alternative UI on top of same data serviceRequestmanipulationIncreasingly Complex constraints=&gt; consider (also) server side validationUsing BeanValidation (1.0 JSR 303 Java EE 6, 1.1 JSR 349 Java EE 8?)Validationhooks in frameworkCustom code
  3. Alternative UI on top of same data serviceRequestmanipulationIncreasingly Complex constraints=&gt; consider (also) server side validationUsing BeanValidation (1.0 JSR 303 Java EE 6, 1.1 JSR 349 Java EE 8?)Validationhooks in frameworkCustom code
  4. Alternative UI on top of same data serviceRequestmanipulationIncreasingly Complex constraints=&gt; consider (also) server side validationUsing BeanValidation (1.0 JSR 303 Java EE 6, 1.1 JSR 349 Java EE 8?)Validationhooks in frameworkCustom code
  5. And Web Application running againstitNumeric Data Type Last Name requiredand no more than 30 charactersCheck Constraint: Age &gt;= 6Check &lt;18 &amp; kids or &gt; 18 and != kidsUK: onesession per attendee per slotUK: onekeynotesession per slot
  6. Alternative:No more plannedsessionattendances are allowedthan the capacity of the room in which the session is scheduledto take placeViolating eventsInsert of attendance (for session) (update of attendancenotallowed, delete cannotviolateconstraint)Update of designated room (of session)Change of room capacity (of room)Let’s focus on first violating event: insert of attendanceCreate post insert trigger that calls functionthatcountsnumber of sessionattendancesandcompareswithsession’s room capacityRaiseexceptionwhennumber &gt; capacityNote: functioncanalsobecalledfrommiddle tier afterposting dataIfafter post (statement &amp; trigger fires) andbeforecommitsomethingsimilar is done in a second session (new attendance, validation) andsubsequentlyboth transactions commit – the end result is invalidNo more plannedsessionattendances are allowedthan the capacity of the room in which the session is scheduledto take placeIfafter post (statement &amp; trigger fires) andbeforecommitsomethingsimilar is done in a second session (new attendance, validation) andsubsequentlyboth transactions commit – the end result is invalidLocking is required!Lock down entire Database – no change, no integrityviolation!Lock AttendancetableToo prohibitiveAlsolock Rooms andSessionstabletoprevent change of session room assignmentand room capacityMore fine grained Lock on SESSION_ATTENDANCE_ROOM_CAPACITY_CONSTRAINT_&lt;RoomIdentifier&gt;&lt;SessionId&gt;For any transaction tryingtoperform a data manipulationthatpotentiallyviolates the SESSION_ATTENDANCE_ROOM_CAPACITY_CONSTRAINT for a certainsession in a certain room, validation/enforcement is required; the fine grainedlockneedstobeacquired; ifitcannotbe, somebodyelse is validatingthisrule for this room/sessioncombination. Whendone, the changes are committedand the sessioncanproceed – withallcommitted changes from the othersession
  7. Against JPASingle thread – constraintenforcedWithout lock – twothreads – constraintby-passedTransaction part 1In Database?Using triggersSame problem!DB trick MVStatement level plus finegrainedlockDemo UK
  8. Suppose UI does not support Update of a SessionAttendanceIf person wants to switch fromonesessiontoanother in a certain slot, (s)he has tocreate a new oneandremove the existingoneIn the right order?!DemonstrateBoth steps are DML statementsConstrainttypicallyenforced at statement levelHowever, sometimestheycanbe made deferred [to transaction commit time]
  9. Implementationfromprevious slideDiscusssynchronization: othersession (i.e. thread) alsomanipulated the collection; does the constraintevaluation cover the complete set of data? Depends on timingTobesure: usesynchronized access To the collection – safe, but slowTosomethingelse? – safe andmuch more scalableand elegant
  10. UK on FirstName, LastNameUpdate existing person to John Smith (S1, T1)Update same person in different session (either name or anotherattribute) =&gt; runs intolockInsert new person John Smith (S2, T2)Runs into a lock! A new Record blockedby a Lock? What is the lock on? Locking on a logical ‘semaphor’ canbedone in a very fine-grained wayFor example: {EMP_NAME_UK1, John, Smith}
  11. Examplewith conferenceRegister as Conference Attendee &amp; PaymentConference registrationshouldonlysucceedwhenpayment is completePaymentshouldnotbedonewhenregistrationfails on some business rule
  12. Common with Web Services and BPELNottrivial at all!If resources are knowntoparticipate in this type of transaction, theycould have a logical state on records (staging, reserved, …) thatrequiresconfirmation (or compensation) tobecometrulyapplied or undone(WS-Transaction is anattempt)