A presentation on AuRegis, and the challenge of establishing a unified electronic system in a federal state. Presentation is about the current efforts to develop a unified electronic business register system in Germany.
Lucene 4.0 is on its way to deliver a tremendous amount of new features and improvements. Beside Real-Time Search & Flexible Indexing DocValues aka. Column Stride Fields is one of the "next generation" features
Esta charla pretende enseñar como añadir RabbitMQ a un proyecto Symfony.
Charla realizada en Symfony Barcelona el dia 2 de julio. Podeis encontrar ejemplos de código en https://github.com/solilokiam/rabbitmqexample
Today, most any application can be “Dockerized.” However, there are special challenges when deploying a distributed application such as Spark on containers. This session will describe how to overcome these challenges in deploying Spark on Docker containers, with many practical tips and techniques for running Spark in a container environment.
Containers are typically used to run stateless applications on a single host. There are significant real-world enterprise requirements that need to be addressed when running a stateful, distributed application in a secure multi-host container environment.
There are decisions that need to be made concerning which tools and infrastructure to use. There are many choices with respect to container managers, orchestration frameworks, and resource schedulers that are readily available today and some that may be available tomorrow including:]
• Mesos
• Kubernetes
• Docker Swarm
Each has its own strengths and weaknesses; each has unique characteristics that may make it suitable, or unsuitable, for Spark. Understanding these differences is critical to the successful deployment of Spark on Docker containers.
This session will describe the work done by the BlueData engineering team to run Spark inside containers, on a distributed platform, including the evaluation of various orchestration frameworks and lessons learned. You will learn how to apply practical networking and storage techniques to achieve high performance and agility in a distributed, container environment.
Speaker
Thomas Phelan, Chief Architect, Blue Data, Inc
Using AI to Build a Self-Driving Query Optimizer with Shivnath Babu and Adria...Databricks
Spark’s Catalyst Optimizer uses cost-based optimization (CBO) to pick the best execution plan for a SparkSQL query. The CBO can choose which join strategy to use (e.g., a broadcast join versus repartitioned join), which table to use as the build side for the hash-join, which join order to use in a multi-way join query, which filter to push down, and others. To get its decisions right, the CBO makes a number of assumptions including availability of up-to-date statistics about the data, accurate estimation of result sizes, and availability of accurate models to estimate query costs.
These assumptions may not hold in real-life settings such as multi-tenant clusters and agile cloud environments; unfortunately, causing the CBO to pick suboptimal execution plans. Dissatisfied users then have to step in and tune the queries manually. In this talk, we will describe how we built Elfino, a Self-Driving Query Optimizer. Elfino tracks each and every query over time—before, during, and after execution—and uses machine learning algorithms to learn from mistakes made by the CBO in estimating properties of the input datasets, intermediate query result sizes, speed of the underlying hardware, and query costs. Elfino can further use an AI algorithm (modeled on multi-armed bandits with expert advice) to “explore and experiment” in a safe and nonintrusive manner using otherwise idle cluster resources. This algorithm enables Elfino to learn about execution plans that the CBO will not consider otherwise.
Our talk will cover how these algorithms help guide the CBO towards better plans in real-life settings while reducing its reliance on assumptions and manual steps like query tuning, setting configuration parameters by trial-and-error, and detecting when statistics are stale. We will also share our experiences with evaluating Elfino in multiple environments and highlight interesting avenues for future work.
Lucene 4.0 is on its way to deliver a tremendous amount of new features and improvements. Beside Real-Time Search & Flexible Indexing DocValues aka. Column Stride Fields is one of the "next generation" features
Esta charla pretende enseñar como añadir RabbitMQ a un proyecto Symfony.
Charla realizada en Symfony Barcelona el dia 2 de julio. Podeis encontrar ejemplos de código en https://github.com/solilokiam/rabbitmqexample
Today, most any application can be “Dockerized.” However, there are special challenges when deploying a distributed application such as Spark on containers. This session will describe how to overcome these challenges in deploying Spark on Docker containers, with many practical tips and techniques for running Spark in a container environment.
Containers are typically used to run stateless applications on a single host. There are significant real-world enterprise requirements that need to be addressed when running a stateful, distributed application in a secure multi-host container environment.
There are decisions that need to be made concerning which tools and infrastructure to use. There are many choices with respect to container managers, orchestration frameworks, and resource schedulers that are readily available today and some that may be available tomorrow including:]
• Mesos
• Kubernetes
• Docker Swarm
Each has its own strengths and weaknesses; each has unique characteristics that may make it suitable, or unsuitable, for Spark. Understanding these differences is critical to the successful deployment of Spark on Docker containers.
This session will describe the work done by the BlueData engineering team to run Spark inside containers, on a distributed platform, including the evaluation of various orchestration frameworks and lessons learned. You will learn how to apply practical networking and storage techniques to achieve high performance and agility in a distributed, container environment.
Speaker
Thomas Phelan, Chief Architect, Blue Data, Inc
Using AI to Build a Self-Driving Query Optimizer with Shivnath Babu and Adria...Databricks
Spark’s Catalyst Optimizer uses cost-based optimization (CBO) to pick the best execution plan for a SparkSQL query. The CBO can choose which join strategy to use (e.g., a broadcast join versus repartitioned join), which table to use as the build side for the hash-join, which join order to use in a multi-way join query, which filter to push down, and others. To get its decisions right, the CBO makes a number of assumptions including availability of up-to-date statistics about the data, accurate estimation of result sizes, and availability of accurate models to estimate query costs.
These assumptions may not hold in real-life settings such as multi-tenant clusters and agile cloud environments; unfortunately, causing the CBO to pick suboptimal execution plans. Dissatisfied users then have to step in and tune the queries manually. In this talk, we will describe how we built Elfino, a Self-Driving Query Optimizer. Elfino tracks each and every query over time—before, during, and after execution—and uses machine learning algorithms to learn from mistakes made by the CBO in estimating properties of the input datasets, intermediate query result sizes, speed of the underlying hardware, and query costs. Elfino can further use an AI algorithm (modeled on multi-armed bandits with expert advice) to “explore and experiment” in a safe and nonintrusive manner using otherwise idle cluster resources. This algorithm enables Elfino to learn about execution plans that the CBO will not consider otherwise.
Our talk will cover how these algorithms help guide the CBO towards better plans in real-life settings while reducing its reliance on assumptions and manual steps like query tuning, setting configuration parameters by trial-and-error, and detecting when statistics are stale. We will also share our experiences with evaluating Elfino in multiple environments and highlight interesting avenues for future work.
PNUTS is a massively parallel and geographically distributed database system for Yahoo!’s web applications. It provides data storage organized as hashed or ordered tables, low latency for large numbers of concurrent requests including updates and queries, and novel per-record consistency guarantees. It is a hosted, centrally managed, and geographically distributed service, and utilizes automated load-balancing and failover to reduce operational complexity. The first version of the system is currently serving in production. This presentation describes the motivation for PNUTS and the design and implementation of its table storage and replication layers, and then presents experimental results.
This is from a 2 hour talk introducing in-memory databases. First a look at traditional RDBMS architecture and some of it's limitations, then a look at some in-memory products and finally a closer look at OrigoDB, the open source in-memory database toolkit for NET/Mono.
En tant que professionnel de l’immobilier, vous devez au quotidien, réunir une masse importante de documents souvent anciens, et éparpillés dans l’entreprise, ce qui peut entraver le bon fonctionnement de vos activités. Il est donc essentiel d’adopter une approche stratégique avec la documentation juridique inhérente au secteur de l’immobilier...
HDFS has several strengths: horizontally scale its IO bandwidth and scale its storage to petabytes of storage. Further it provides very low latency metadata operations and scales to over 60K concurrent clients. One of its limitations is scaling number of files and blocks in the system. We describe a radical change to Hadoop’s storage infrastructure with the upcoming Ozone technology. It allows Hadoop to scale to tens of billions of files and blocks and, in the future, to every larger number of smaller objects. Ozone fundamentally separates the namespace layer and the block layer allowing new namespace layers to be added in the future. Further, the use of RAFT protocol has allowed the storage layer to be self-consistent. It also means that the HA solution for the Metadata server (the Keyspace manager/future NN). We show how this technology helps a Hadoop user and also what it means for evolving HDFS in the future. We will also cover the technical details of Ozone.
Speaker
Sanjay Radia, Chief Architect and Founder, Hortonworks
Finite state automata and transducers made it into Lucene fairly recently, but already show a very promising impact on search performance. This data structure is rarely exploited because it is commonly (and unfairly) associated with high complexity. During the talk, I will try to show that automata and transducers are in fact very simple, their construction can be very efficient (memory and time-wise) and their field of applications very broad.
The CRF Innovation Awards celebrate the spirit of innovation and honours CRF jurisdictions which have successfully put in place world-class initiatives, as well as those which have creatively implemented customised solutions, to make a real difference to their stakeholders. There are two categories of awards: CRF Innovation Award (Excellence) and CRF Innovation Award (Commendation).
The CRF Innovation Awards celebrate the spirit of innovation and honours CRF jurisdictions which have successfully put in place world-class initiatives, as well as those which have creatively implemented customised solutions, to make a real difference to their stakeholders. There are two categories of awards: CRF Innovation Award (Excellence) and CRF Innovation Award (Commendation).
Slovenia - AJPES Digitisation a more transparent non possessory lien rights r...Corporate Registers Forum
The CRF Innovation Awards celebrate the spirit of innovation and honours CRF jurisdictions which have successfully put in place world-class initiatives, as well as those which have creatively implemented customised solutions, to make a real difference to their stakeholders. There are two categories of awards: CRF Innovation Award (Excellence) and CRF Innovation Award (Commendation).
The CRF Innovation Awards celebrate the spirit of innovation and honours CRF jurisdictions which have successfully put in place world-class initiatives, as well as those which have creatively implemented customised solutions, to make a real difference to their stakeholders. There are two categories of awards: CRF Innovation Award (Excellence) and CRF Innovation Award (Commendation).
PNUTS is a massively parallel and geographically distributed database system for Yahoo!’s web applications. It provides data storage organized as hashed or ordered tables, low latency for large numbers of concurrent requests including updates and queries, and novel per-record consistency guarantees. It is a hosted, centrally managed, and geographically distributed service, and utilizes automated load-balancing and failover to reduce operational complexity. The first version of the system is currently serving in production. This presentation describes the motivation for PNUTS and the design and implementation of its table storage and replication layers, and then presents experimental results.
This is from a 2 hour talk introducing in-memory databases. First a look at traditional RDBMS architecture and some of it's limitations, then a look at some in-memory products and finally a closer look at OrigoDB, the open source in-memory database toolkit for NET/Mono.
En tant que professionnel de l’immobilier, vous devez au quotidien, réunir une masse importante de documents souvent anciens, et éparpillés dans l’entreprise, ce qui peut entraver le bon fonctionnement de vos activités. Il est donc essentiel d’adopter une approche stratégique avec la documentation juridique inhérente au secteur de l’immobilier...
HDFS has several strengths: horizontally scale its IO bandwidth and scale its storage to petabytes of storage. Further it provides very low latency metadata operations and scales to over 60K concurrent clients. One of its limitations is scaling number of files and blocks in the system. We describe a radical change to Hadoop’s storage infrastructure with the upcoming Ozone technology. It allows Hadoop to scale to tens of billions of files and blocks and, in the future, to every larger number of smaller objects. Ozone fundamentally separates the namespace layer and the block layer allowing new namespace layers to be added in the future. Further, the use of RAFT protocol has allowed the storage layer to be self-consistent. It also means that the HA solution for the Metadata server (the Keyspace manager/future NN). We show how this technology helps a Hadoop user and also what it means for evolving HDFS in the future. We will also cover the technical details of Ozone.
Speaker
Sanjay Radia, Chief Architect and Founder, Hortonworks
Finite state automata and transducers made it into Lucene fairly recently, but already show a very promising impact on search performance. This data structure is rarely exploited because it is commonly (and unfairly) associated with high complexity. During the talk, I will try to show that automata and transducers are in fact very simple, their construction can be very efficient (memory and time-wise) and their field of applications very broad.
The CRF Innovation Awards celebrate the spirit of innovation and honours CRF jurisdictions which have successfully put in place world-class initiatives, as well as those which have creatively implemented customised solutions, to make a real difference to their stakeholders. There are two categories of awards: CRF Innovation Award (Excellence) and CRF Innovation Award (Commendation).
The CRF Innovation Awards celebrate the spirit of innovation and honours CRF jurisdictions which have successfully put in place world-class initiatives, as well as those which have creatively implemented customised solutions, to make a real difference to their stakeholders. There are two categories of awards: CRF Innovation Award (Excellence) and CRF Innovation Award (Commendation).
Slovenia - AJPES Digitisation a more transparent non possessory lien rights r...Corporate Registers Forum
The CRF Innovation Awards celebrate the spirit of innovation and honours CRF jurisdictions which have successfully put in place world-class initiatives, as well as those which have creatively implemented customised solutions, to make a real difference to their stakeholders. There are two categories of awards: CRF Innovation Award (Excellence) and CRF Innovation Award (Commendation).
The CRF Innovation Awards celebrate the spirit of innovation and honours CRF jurisdictions which have successfully put in place world-class initiatives, as well as those which have creatively implemented customised solutions, to make a real difference to their stakeholders. There are two categories of awards: CRF Innovation Award (Excellence) and CRF Innovation Award (Commendation).
The CRF Innovation Awards celebrate the spirit of innovation and honours CRF jurisdictions which have successfully put in place world-class initiatives, as well as those which have creatively implemented customised solutions, to make a real difference to their stakeholders. There are two categories of awards: CRF Innovation Award (Excellence) and CRF Innovation Award (Commendation).
The CRF Innovation Awards celebrate the spirit of innovation and honours CRF jurisdictions which have successfully put in place world-class initiatives, as well as those which have creatively implemented customised solutions, to make a real difference to their stakeholders. There are two categories of awards: CRF Innovation Award (Excellence) and CRF Innovation Award (Commendation).
A presentation on the role of data and users in the experience of the Labuan International Business Finance Centre. In particular the registry application.
Challenges in Modern Registry Management - US persceptive.
Germany - AuRegisThe challenge of establishing a unified electronic system in a federal state
1. Die Justiz des Landes
Nordrhein-Westfalen
www.justiz.nrw
AuRegis
The challenge of establishing a unified electronic system in
a federal state
Skopje, 09.04.2019
Nicolas Moos, Verfahrenspflegestelle RegisSTAR, 09.04.2019 1
2. Die Justiz des Landes
Nordrhein-Westfalen
www.justiz.nrw
The Current Situation
2
16 States
114 Courts
2 Business Register Systems
RegisSTAR
Aureg
Nicolas Moos, Verfahrenspflegestelle RegisSTAR, 09.04.2019
3. Die Justiz des Landes
Nordrhein-Westfalen
www.justiz.nrw
Requirements
2 IT-Infrastructures
Reference Environment 1: Oracle Stack
Reference Environment 2: Microsoft Stack
3 different systems for filing
e2A
eIP
eAS
3Nicolas Moos, Verfahrenspflegestelle RegisSTAR, 09.04.2019
4. Die Justiz des Landes
Nordrhein-Westfalen
www.justiz.nrw
Requirements
Service-Oriented Architecture (SoA)
4Nicolas Moos, Verfahrenspflegestelle RegisSTAR, 09.04.2019
5. Die Justiz des Landes
Nordrhein-Westfalen
www.justiz.nrw
Requirements
Building Software Components in coorperation with the
German Land Register
Serving different File Management Systems
Serving different Infrastructure Components
Accessibility (BITV 2.0)
5Nicolas Moos, Verfahrenspflegestelle RegisSTAR, 09.04.2019
6. Die Justiz des Landes
Nordrhein-Westfalen
www.justiz.nrw
Achievements
6Nicolas Moos, Verfahrenspflegestelle RegisSTAR, 09.04.2019
Professional and technical exchange
Cross-state core teams
Lead Management NRW
Bundled communication with service provider
7. Die Justiz des Landes
Nordrhein-Westfalen
www.justiz.nrw
Achievements
7Nicolas Moos, Verfahrenspflegestelle RegisSTAR, 09.04.2019
Regular project meetings
Continuous test camps
Introduction of IT governance
Core system of AuRegis is almost completed
8. Die Justiz des Landes
Nordrhein-Westfalen
www.justiz.nrw
Challenges
8Nicolas Moos, Verfahrenspflegestelle RegisSTAR, 09.04.2019
Different version levels
Different components
Cross-state coordination
9. Die Justiz des Landes
Nordrhein-Westfalen
www.justiz.nrw
Challenges
9Nicolas Moos, Verfahrenspflegestelle RegisSTAR, 09.04.2019
Different service providers
Different projects involved
Gaps in interaction
Moving targets
10. Die Justiz des Landes
Nordrhein-Westfalen
www.justiz.nrw
Possible Reasons
10Nicolas Moos, Verfahrenspflegestelle RegisSTAR, 09.04.2019
Many players involved
Federalism
Unity on a large scale, disunity on details
Multilateral coordination
11. Die Justiz des Landes
Nordrhein-Westfalen
www.justiz.nrw
Possible Reasons
11Nicolas Moos, Verfahrenspflegestelle RegisSTAR, 09.04.2019
Interdependence
High Aims/ Complexity
12. Die Justiz des Landes
Nordrhein-Westfalen
www.justiz.nrw
Next Steps
12Nicolas Moos, Verfahrenspflegestelle RegisSTAR, 09.04.2019
Data Migration
Integration
Piloting
Commissioning
13. Die Justiz des Landes
Nordrhein-Westfalen
www.justiz.nrw13
Thank you for your attention
Questions?
Nicolas Moos, Verfahrenspflegestelle RegisSTAR, 09.04.2019
Hinweis der Redaktion
Presentation is about the current efforts to develop a unified electronic business register system in Germany.
Why does this topic fit into this working session?
Because Germany is a federal state...
Germany consists of 16 federal states. Each of these federal states is responsible for maintaining the business register. The business register itself is run by our local courts. There is a total number of 114 local courts that run the business register.
Since 2007, the business register has been operated electronically throughout Germany. For this purpose, two different electronic systems developed at that time, as you can see on the map.
In 12 federal states the system "RegisSTAR" has been implemented and in 4 federal states the other system called "Aureg". In 2011 the Aureg states decided to join the RegisSTAR system and it was agreed to develop a new united system to replace the existing ones. This new system is called Auregis. We have been working on it since 2014.
What are the requirements for AuRegis?
The new system must be compatible with two different IT infrastructures and be able to serve three different file systems.
In addition, we intend to develop the software on a modular basis in a so-called service-oriented architecture.
This enables individual components of the software to be used for other systems as well.
Some of these components will also be used by the land registry in the future.
In addition, AuRegis must be able to serve various file management systems and various infrastructure components and be barrier-free.
_______________________________________
User Authorization Administration
Management of personal data
Document Generation
Logging
Time Limit Administration
Cost & Fee Processing
Allocation of Duties Administration
What have we achieved so far?
We have set up three core teams made up of members of the judiciary from various federal states and of staff from our contracted service provider.
The core teams coordinate which functionalities are required from a technical point of view and how these can be implemented technically.
Regular online conferences and workshops are held for this purpose.
Communication is coordinated by NRW. If fundamental changes are about to arise or new contracts have to be awarded, NRW will bundle discussions with the service provider.
After the completion of the individual project phases, there are face-to-face meetings in which members of the judiciary from all federal states can test the work of the service provider.
In order to coordinate the various projects (e.g. business register and land register), the project managers of all participating projects meet regularly.
In order to improve decision-making processes at the technical level, IT governance is to be introduced. It has to be said that the introduction is at the very beginning, so that the effects cannot yet be estimated.
The achievements described above have led to the fact that the core system of AuRegis has now been completed with a few exceptions.
Now the system is still not in operation. This brings us to the next topic - the challenges that arise in a federal system.
There are different components in the different federal states that all have to be connected.
In some cases, the versions of the software used in the different federal states are on different levels, so that they are not compatible with each other.
Cross-state coordination can be difficult at the professional level.
In addition, different projects are involved when it comes to the use of the basic components. The different projects have different ideas or change their ideas over time.
In the different federal states, different service providers are commissioned, so that they have to work together, which does not always happen smoothly.
In the interaction of the core system with the file systems and the text systems, unexpected gaps have arisen which lead to delays (e.g. the sending of messages can take place from AuRegis , from the eFile or from the text system). Which component is to perform this task is still unclear).
As a result of such planning deviations, new targets need to be set in every project and these new targets must be reconciled again.
Due to the federal system, different structures have grown, which must be united with each other.
There are a lot of people from different federal states and different projects involved, which makes coordination rather tricky.
Even if one agrees on the result, there is often disagreement about the right way to reach the goal. These disagreements only become apparent in the detailed planning.
When ideas conflict, there is no real hierarchy. It is rather necessary to reach agreements with many sides and to work towards a win-win situation so that a common ground can be found.
The targets are quite ambitious, as the requirements for the new system are extremely complex.
Due to the interdependence of the individual components, a setback in a single area can lead to a delay in the overall project.
Our next steps will be to merge the individual pieces of the puzzle into one big whole.
Then the data has to be migrated from the old systems.
The new system will then be piloted at six different courts in different federal states. Piloting is planned for next year.
After successful piloting, the system will then be used nationwide.