Using software modules welcome to hell!

•Als PPTX, PDF herunterladen•

2 gefällt mir•891 views

Using software modules today is the default way of working for most systems and frameworks. With the advent of many software languages and OSS frameworks, new module systems are constantly created and new module ecosystems start to prevail. This trend is horizontal and covers operating system packages, language libraries and application modules (plugins). But while some module systems are nicer to use, others are repeating past mistakes and are a daily source for developer agony and pain. In this short talk I will present the \"lessons learned\" at JFrog, where we make software for managing software libraries and deal with many types of module systems. This talk will show what works and what doesn\'t work in a module system; what features can make a module ecosystem thrive or fail; and why, despite all downsides, modules are here to stay and conquer more space as the Cloud continues to grow.

Technologie Business

Using Software Modules -
Welcome to Hell!

Agenda
25 minutes
125 slides
30 memes

Packages & Modules
Modules Packages
API
development

Packages & Modules
Modules Packages
API
development
Installable
runtime

Modules, modules,
modules
System dpkg, RPM, tgz, pacman, ipkg, pkgutils, PETget,
Upkg, PISI, nix, Equo, Conary, tazpkg, App Store,
MacPorts, fink, Homebrew, Google Play, GetJar,
Amazon Appstore, Cygwin, Npackd, Steam,
Chocolately, NSIS, IPS

Modules, modules,
modules
System dpkg, RPM, App Store,
Homebrew, Google Play,
Steam

Modules, modules,
modules
System dpkg, RPM, App Store,
Homebrew, Google Play,
Steam
Dev. Cabal, Composer, CPAN, CRAN, Python eggs, Ivy,
Maven, Gradle, sbt, leiningen, PyPI, NuGet,
RubyGems, Quicklisp, CocoaPods

Modules, modules,
modules
System dpkg, RPM, App Store,
Homebrew, Google Play,
Steam
Dev. Ivy, Maven, Gradle, sbt,
PyPI, NuGet, RubyGems,
CocoaPods

Modules, modules,
modules
System dpkg, RPM, App Store,
Homebrew, Google Play,
Steam
Dev. Ivy, Maven, Gradle, sbt,
PyPI, NuGet, RubyGems,
CocoaPods
Framewor
k
RoR, Grails, jQuery, Node.js, Spring Roo, JBoss
Forge, Play, Chef, Puppet, Eclipse

, A STORY OF INDEXES +
< KEPT IN WRONG PLACE >

Once Upon A Time…
Can’t
serve…
searches…

- Have to download huge files
Before Searching

- Have to download huge files
Before Searching
- Updated rarely

- Have to download huge files
Before Searching
- Updated rarely
- Requires special client

Who’s good, who’s not?
NuGet
RubyGem
s
Deb
RPM
Maven

Who’s good, who’s not?
NuGet
RubyGem
s
Deb
RPM
Maven
Good
Excuse:
old tools

Lesson learned
Index on the server-
side,
expose query API

He cut his ear off
and died unappreciated

Lesson learned
Authenticate by public
identity;
Give credit to the
developer

Depending on name only
Where’s the
version,
dude?!

TIME FOR THE BOROMIR
MEME!
You knew it is coming!

Single Namespace Troubles
Scope Troubl
e
Examples Work-
around

Single Namespace Troubles
Scope Troubl
e
Examples Work-
around
System Can’t
use
multipl
e
version
s,
update
all

Single Namespace Troubles
Scope Troubl
e
Examples Work-
around
System Can’t
use
multipl
e
version
s,
update
all
RPMs,
Ruby,
Groovy

Single Namespace Troubles
Scope Troubl
e
Examples Work-
around
System Can’t
use
multipl
e
version
s,
update
all
RPMs,
Ruby,
Groovy
RVM, GVM,
Homebre
w

Single Namespace Troubles
Scope Troubl
e
Examples Work-
around
System Can’t
use
multipl
e
version
s,
update
all
RPMs,
Ruby,
Groovy
RVM, GVM,
Homebre
w
Platfor Classpa

Maven’s Conflict Manager
A v1
B v1E v1
D v1

Maven’s Conflict Manager
A v1
C v1
B v1E v1
D v1

Maven’s Conflict Manager
A v1
C v1
B v1
D v2
E v1
D v1

Lesson learned
Provide isolation and
easy uninstall

Lesson learned
Require explicit
dependency versions

My
Project
‘A1’
<alwaysUse>
<repository>MyRepo</repository>
</alwaysUse>
MyRepo

My
Project
‘A1’
<dependency>B1</dependency>
<alwaysUse>
<repository>MyRepo</repository>
</alwaysUse>
MyRepo

My
Project
‘A1’
<dependency>B1</dependency>
<alwaysUse>
<repository>MyRepo</repository>
</alwaysUse>
MyRepo
Project
‘B1’

My
Project
‘A1’
<dependency>B1</dependency>
<alwaysUse>
<repository>MyRepo</repository>
</alwaysUse>
MyRepo
Project
‘B1’
BadRepo
<repository>BadRepo</repository>
<dependency>C1</dependency>


My
Project
‘A1’
<dependency>B1</dependency>
<dependency>C1</dependency>
<repository>MyRepo</repository>
<repository>BadRepo</repository>
MyRepo
Project
‘B1’
BadRepo

Lesson learned
Don’t mix vodka and
beer configuration and
metadata

NO MORE STORIES,
ONE MORE THINGOOO
fACDdb

Weitere ähnliche Inhalte

Was ist angesagt?

Heroku + Jeweler & Gemcutter

Ariejan de Vroom

We can be brilliant developers, but we won’t succeed—and won’t lead our organizations to succeed—without a new perspective (if you will) and new assumptions about the components of the “technology ecosystem” that are fundamentally critical to our success. This includes the operators, QA team, DBAs, security folks, and even the pure business contingent—in most cases, each of these individuals and groups plays a critical role in the success of what we create and give birth to as developers. What we do in isolation might be genius, but if we insulate ourselves—especially with arrogance—from these colleagues, neither our code nor our organizations will realize their full potential, and most will fail. The bottom line is that our old ways are no longer viable, and as the elite within our industry, we will be the leaders and heroes who discard old assumptions and adopt a new perspective in this exciting journey to digital transformation—where the impossible can become reality.

Teaching Elephants to Dance (Federal Audience): A Developer's Journey to Digi...

Burr Sutter

Super-powered CI with Git - Sarah Goff-Dupont

Atlassian

Ansible, Idempotency, and Jenkins

tylerturk

Bridging the Gap - Laracon 2013

Ben Corlett

Perl Development Environment Tooling

Jason Crome

Stackato v6

Jonas Brømsø

Love it or hate it (and a lot of people seem to hate it), Maven is a widely used tool. We can consider that Maven has been the de-facto standard build tool for Java over the last 10 years. Most experienced developers already got their share of Maven headaches. Unfortunately, new developers are going through the same hard learning process, because they don't know how to deal with Maven particularities. "Why is this jar in my build?", "I can’t see my changes!", "The jar is not included in the distribution!", "The artifact was not found!" are common problems. Learn to tame the Maven Beast and be in complete control of your build to save you countless hours of pain and frustration.

Maven - Taming the Beast

Roberto Cortez

Are you still stuck in Java EE 5? Eager to move and boost developer productivity with all the cool things introduced in Java EE 7? Attend this session to hear about some of the solutions Tomitribe had to implement to completely migrate an application called Segurnet from Java EE 5 to Java EE 7. Expect a very technical session that delves into the details. Segurnet is a platform held by APS (Portuguese Insurance Association) that has served as an integration network for the insurance sector in Portugal for the last 20 years, with more than 33,000 active users.

Migration tales from java ee 5 to 7

Roberto Cortez

DevTernity - DevOps with smell

Antons Kranga

With more businesses moving to cloud-based solutions everyday, we must re-think the strategies used to deploy Perl applications and related libraries, given the volatile aspects of the cloud and its constraints. In this talk I go over the challenges posed by virtualised environments, and consider several solutions to them. The use cases are all related to Amazon's EC2, but will easily be adapted for GoGrid, Mosso, and others.

Perl in Teh Cloud

Pedro Figueiredo

Composer - The missing package manager for PHP

Tareq Hasan

Your Goat Anti-Fragiled My Snowflake! Demystifying DevOps Jargon (30 minute v...

Clinton Wolfe

Building REST APIs using gRPC and Go

Alvaro Viebrantz

Perl-Critic

Jonas Brømsø

Php Dependency Management with Composer ZendCon 2016

Clark Everetts

ZN-2015

Ivan Elkin

Testing your puppet code

Julien Pivotto

Configuration as Code in Bamboo

Atlassian

Building Open-source React Components

Zack Argyle

Was ist angesagt? (20)

Heroku + Jeweler & Gemcutter

Teaching Elephants to Dance (Federal Audience): A Developer's Journey to Digi...

Super-powered CI with Git - Sarah Goff-Dupont

Ansible, Idempotency, and Jenkins

Bridging the Gap - Laracon 2013

Perl Development Environment Tooling

Stackato v6

Maven - Taming the Beast

Migration tales from java ee 5 to 7

DevTernity - DevOps with smell

Perl in Teh Cloud

Composer - The missing package manager for PHP

Your Goat Anti-Fragiled My Snowflake! Demystifying DevOps Jargon (30 minute v...

Building REST APIs using gRPC and Go

Perl-Critic

Php Dependency Management with Composer ZendCon 2016

ZN-2015

Testing your puppet code

Configuration as Code in Bamboo

Building Open-source React Components

Ähnlich wie Using software modules welcome to hell!

Ruby - The Hard Bits

Paul Gallagher

Testing Adhearsion Applications

Luca Pradovera

PHP is one of the easiest programming languages to use ever and powers more than half of the internet. With this ease of use, certain common patterns emerge that become harmful. This is especially true when your product or service is not expected to die soon. Some anti-patterns are coding, others are related to operating your service, especially with new docker stacks. We will go over some of the most common pitfalls with a focus on enterprise development.

PHP Mega Meetup, Sep, 2020, Anti patterns in php

Ahmed Abdou

Python+gradle

Stephen Holsapple

Deep dive into Verdaccio - NodeTLV 2022 - Israel

Juan Picado

ASP.NET 5 auf Raspberry PI & docker

Jürgen Gutsch

This talk will try to cover the most important techniques and best practices used when creating Django web application. Overview of the topics covered: - development general principles and goals - python/django project initial setup - project layout, git&venv&pip&shell, settings - central project shell command - contains all commands to manage project - "IDE" - editor & shell - edit/run/test cycle - deploy/test-remotely cycle Disclaimer: techniques and practices presented are current AUTHOR'S optimal choice used for usual django project.

Django dev-env-my-way

Robert Lujo

Fish Cam.pptx

PhilipBasford

Ransack, an Application Built on Ansible's API for Rackspace -- AnsibleFest N...

Paul Durivage

Puppet for SysAdmins

Puppet

As part of the 2014 USENIX Release Engineering Summit West, I presented a talk about packaging software and what's wrong with current trends. Here's the abstract: Reliably distributing software is a notoriously difficult problem, and almost every operating system and programming language vendor has tried to solve it. This has led to a herd of packaging systems, almost none of which are cross-compatible; some manage system-level software, while others focus on extending their own language (often by trampling on system-level software). And like all competing standards, every packaging system comes with its own sharp corners, dull edges, and hidden idiosyncrasies to deal with along the path to packaging happiness. In an attempt to answer the question "How do I install this software and ensure that its dependencies are fulfilled?", some novel solutions have begun to see popular adoption. But a lot of these newer tools and techniques tread the same ground as their predecessors while overlooking the lessons that were learned along the way. I'll talk about the state of native packaging systems on some popular platforms (Debian/Ubuntu, RHEL/CentOS/Fedora, and Mac OS X), packaging systems for popular languages (Ruby, Python, Perl, and Node) and the ways that developers are attempting to work around the limitations of these systems. I'll review the reasons that tools like curlbash, FPM, and omnibus packages have become popular by sharing lessons I've learned while working through these systems. While this will be an amusing presentation, I'll show how native packages can address the concerns that have pushed Release Engineers and Developers away. I will also talk about what native packaging systems can learn from the next generation of packaging tools. The original abstract is available here: https://www.usenix.org/conference/ures14west/summit-program/presentation/mckern

Packaging is the Worst Way to Distribute Software, Except for Everything Else

mckern

Django: Beyond Basics

arunvr

I gave a talk titled "Continuous Integration in data centers“ at OSDC in 2013, presenting ways how to realize continuous integration/delivery with Jenkins and related tools.Three years later we gained new tools in our continuous delivery pipeline, including Docker, Gerrit and Goss. Over the years we also had to deal with different problems caused by faster release cycles, a growing team and gaining new projects. We therefore established code review in our pipeline, improved our test infrastructure and invested in our infrastructure automation.In this talk I will discuss the lessons we learned over the last years, demonstrate how a proper continuous delivery pipeline can improve your life and how open source tools like Jenkins, Docker and Gerrit can be leveraged for setting up such an environment.

OSDC 2016 - Continous Integration in Data Centers - Further 3 Years later by ...

NETWAYS

Approaching package manager

Timur Safin

Hadoop is everywhere these days, but it can seem like a complex, intimidating ecosystem to those who have yet to jump in. In this hands-on workshop, Alex Dean, co-founder of Snowplow Analytics, will take you "from zero to Hadoop", showing you how to run a variety of simple (but powerful) Hadoop jobs on Elastic MapReduce, Amazon's hosted Hadoop service. Alex will start with a no-nonsense overview of what Hadoop is, explaining its strengths and weaknesses and why it's such a powerful platform for data warehouse practitioners. Then Alex will help get you setup with EMR and Amazon S3, before leading you through a very simple job in Pig, a simple language for writing Hadoop jobs. After this we will move onto writing a more advanced job in Scalding, Twitter's Scala API for writing Hadoop jobs. For our final job, we will consolidate everything we have learnt by building a more sophisticated job in Scalding.

From Zero to Hadoop: a tutorial for getting started writing Hadoop jobs on Am...

Alexander Dean

The Secrets of The FullStack Ninja - Part A - Session I

Oded Sagir

Puppet for Sys Admins

Puppet

Pilot Tech Talk #10 — Practical automation by Kamil Cholewiński

Pilot

Tiny Puppet Can Install Everything. Prove me wrong!

Alessandro Franceschi

Bettercap

Rajivarnan (Rajiv)

Ähnlich wie Using software modules welcome to hell! (20)

Ruby - The Hard Bits

Testing Adhearsion Applications

PHP Mega Meetup, Sep, 2020, Anti patterns in php

Python+gradle

Deep dive into Verdaccio - NodeTLV 2022 - Israel

ASP.NET 5 auf Raspberry PI & docker

Django dev-env-my-way

Fish Cam.pptx

Ransack, an Application Built on Ansible's API for Rackspace -- AnsibleFest N...

Puppet for SysAdmins

Packaging is the Worst Way to Distribute Software, Except for Everything Else

Django: Beyond Basics

OSDC 2016 - Continous Integration in Data Centers - Further 3 Years later by ...

Approaching package manager

From Zero to Hadoop: a tutorial for getting started writing Hadoop jobs on Am...

The Secrets of The FullStack Ninja - Part A - Session I

Puppet for Sys Admins

Pilot Tech Talk #10 — Practical automation by Kamil Cholewiński

Tiny Puppet Can Install Everything. Prove me wrong!

Bettercap

Mehr von Baruch Sadogursky

So, you want to update the software for your user, be it the nodes in your K8s cluster, a browser on user’s desktop, an app in user’s smartphone or even a user’s car. What can possibly go wrong? In this talk, we’ll analyze real-world software update fails and how multiple DevOps patterns, that fit a variety of scenarios, could have saved the developers. Manually making sure that everything works before sending an update and expecting the user to do acceptance tests before they update is most definitely not on the list of such patterns. Join us for some awesome and scary continuous update horror stories and some obvious (and some not so obvious) proven ideas for improvement and best practices you can start following tomorrow.

DevOps Patterns & Antipatterns for Continuous Software Updates @ NADOG April ...

Baruch Sadogursky

DevOps Patterns & Antipatterns for Continuous Software Updates @ DevOps.com A...

Baruch Sadogursky

In this talk, we’ll take you to a scaling journey, from 3 developers to a 100. We’ll talk about the challenges each milestone in this growth brings, both technological and methodological, and how to solve those challenges using the right mix of people, the right selection of tools and the correctly crafted process. The speakers excel in the different aspects of this triangle and went through this journey (more than once) themselves. And the fun and entertaining presentation as a Greek tragedy can’t hurt, can it?

DevOps @Scale (Greek Tragedy in 3 Acts) as it was presented at Oracle Code NY...

Baruch Sadogursky

Devops is usually viewed from a traditional perspective of a collaboration of Dev, Ops, and QA, driven by the change in Culture, People, and Process. But how do you know where you stand and where to move? As in almost any field, data and metrics give you the gauges and instruments. In this talk, we’ll talk about the key measurements for the DevOps transformation process and provide you with 3 metrics you can start measuring tomorrow.

Data driven devops as presented at QCon London 2018

Baruch Sadogursky

A Research Study Into DevOps Bottlenecks as presented at Oracle Code LA 2018

Baruch Sadogursky

Java Puzzlers NG S03 a DevNexus 2018

Baruch Sadogursky

Do you always know what’s going on with your product artifacts since the moment they are built by the CI server from Git sources all the way to being deployed by Helm into Kuberenetes? In this talk, we will show how to build a reliable and transparent pipeline from code to cluster using Git, Artifactory, Docker, Kubernetes, and Helm. We’ll show how you such a pipeline can help you answer the big questions: What to deploy, What is deployed, and what is this artifact that I am looking for. This kind of transparency is critical for today’s environments, and Kubernetes with Helm shouldn’t be an exception.

Where the Helm are your binaries? as presented at Canada Kubernetes Meetups

Baruch Sadogursky

By Baruch Sadogursky Devops is usually viewed from a traditional perspective of a collaboration of Dev, Ops and QA, driven by the change in Culture, People and Process. But how do you know where you stand and were to move? As in almost any field, data and metrics give you the gauges and instruments. In this talk we’ll talk about the key measurements for the DevOps transformation process and provide you with 3 metrics you can start measuring tomorrow.

Data driven devops as presented at Codemash 2018

Baruch Sadogursky

A Research Study into DevOps Bottlenecks as presented at Codemash 2018

Baruch Sadogursky

By Baruch Sadogursky There are three hard things in computer science: cache invalidation, naming things, and off-by-one errors. This session tackles naming, especially Docker version naming. Labels, tags, checksums...how should you use them to keep track of Docker versions? What about dev versus prod images—how best to distinguish those? What about the “latest” tag? What about cleanup? Could we do more? Versioning often seems like a simple problem, but when you have a tool that gives you as much power and flexibility as Docker does, it often helps to develop guidelines. The presentation examines the tools available for managing Docker images and some simple patterns you can employ in various use cases for CI/CD to keep track of your containers.

Best Practices for Managing Docker Versions as presented at JavaOne 2017

Baruch Sadogursky

Debugging applications in production is like being the detective in a crime movie. Especially with microservices. Especially with containers. Especially in the cloud. Trying to see what’s going on in a production deployment at scale is impossible without proper tools! Google has spent over a decade deploying containerized Java applications at unprecedented scale and the infrastructure and tools developed by Google have made it uniquely possible to manage, troubleshoot, and debug, at scale. Join this session to see how you can diagnose and troubleshoot production issues w/ out of the box Kubernetes tools, as well as getting insight from the ecosystem with Weave Scope, JFrog Artifactory & Stackdriver tools.

Troubleshooting & Debugging Production Microservices in Kubernetes as present...

Baruch Sadogursky

DevOps @Scale (Greek Tragedy in 3 Acts) as it was presented at Devoxx 2017

Baruch Sadogursky

Amazon Alexa Skills vs Google Home Actions, the Big Java VUI Faceoff as prese...

Baruch Sadogursky

DevOps @Scale (Greek Tragedy in 3 Acts) as it was presented at DevOps Days Be...

Baruch Sadogursky

Moar puzzlers! The more we work with Java 8, the more we go into the rabbit hole. Did they add all those streams, lambdas, monads, Optionals and CompletableFutures only to confuse us? It surely looks so! And Java 9 that heads our way brings even more of what we like the most, more puzzlers, of course! In this season we as usual have a great batch of the best Java WTF, great jokes to present them and great prizes for the winners!

Java Puzzlers NG S02: Down the Rabbit Hole as it was presented at The Pittsbu...

Baruch Sadogursky

DevOps @Scale (Greek Tragedy in 3 Acts) as it was presented at The Pittsburgh...

Baruch Sadogursky

Let’s Wing It: A Study in DevRel Strategy

Baruch Sadogursky

In this talk, Baruch Sadogursky presents the challenges of a high demand SaaS product incident triage at scale, as well as discuss the sources of log items, including the platform, tenants and other types of log sources. He will show practical examples of collector and filters configuration and will take you through a number of real world examples of problems investigations using Artifactory and Sumo Logic.

Log Driven First Class Customer Support at Scale

Baruch Sadogursky

No relationship in DevOps is more important than that between your CI/CD server and your Binary Repository. Jenkins has long been the go-to server for CI/CD, and JFrog Artifactory has long been one of the most popular integrations with it. This webinar focuses on the new features of the integration, leveraging the Jenkins Pipeline DSL for infrastructure-as-code of your favorite artifactory features whether it be generic, maven, gradle or Docker, and will show an end-to-end example of pipelines across multiple technologies and how powerful these new capabilities are.

[Webinar] The Frog And The Butler: CI Pipelines For Modern DevOps

Baruch Sadogursky

While Docker has enabled an unprecedented velocity of software production, it is all too easy to spin out of control. A promotion-based model is required to control and track the flow of Docker images as much as it is required for a traditional software development lifecycle. New tools often introduce new paradigms. We will examine the patterns and the antipatterns for Docker image management, and what impact the new tools have on the battle-proven paradigms of the software development lifecycle.

Patterns and antipatterns in Docker image lifecycle as was presented at DC Do...

Baruch Sadogursky

Mehr von Baruch Sadogursky (20)