What's New in Globus - Internet2 TechEXtra

•

0 gefällt mir•226 views

An overview of developments in the Globus platform during 2020-2021, presented at a webinar hosted by Internet2. Includes an overview of Globus Connect Server v5, cloud storage connectors, and platform services for developers (e.g., Globus Search and Globus Flows).

Technologie

What’s New
in Globus
Vas Vasiliadis – vas@uchicago.edu
Internet2 TechEXtra – February 23, 2022

What a difference three years make!
We’ve been busy…
• …Rebuilding the technical foundation
• …Growing the connected storage ecosystem
• …Expanding the data management platform
• …Automating data management at scale
3

Rebuilding the technical foundation: GCSv5
• Modern security model
• Support compliance requirements
• Deployment flexibility
• Enhanced sharing policies
• Simplified scaling/availability
4

Data-appropriate storage
Google Drive for project admin files
Google Cloud Storage for core research files
Reliably manage API constraints

Support for all major cloud
storage providers

Partnership with the community
to develop new connectors
Community Connector Program

Easy egress and ingress of data
Data sharing with collaborators
via unified interface

POSIX Staging Connector
• Example:
– IBM Spectrum Archive plugin, Brock Palen at University of
Michigan - github.com/brockpalen/ltfsee-globus
12
• For POSIX file systems
that cache from tertiary
storage
• Custom plug-in for
staging files

Streamlined GCP Installation
• Log in from the
application during
installation
• Use setup key method
for automation

New platform services
• Groups
• Search
• Flows
14

Globus Groups
docs.globus.org/api/groups
groups.api.globus.org/redoc
15

Globus Search: Data description/discovery
• Metadata store with fine-
grained visibility controls
• Schema agnostic
à dynamic schemas
• Simple search using URL
query parameters
• Complex search using
search request document
• Example:
16
docs.globus.org/api/search
Search
Index
github.com/globus/searchable-files-demo

Globus Search
Terra Fusion on OSN
17
osn.globusdemo.org

3 Degrees of Automation
Timer Service
Scheduled and recurring transfers
(a.k.a. Globus cron)
Command Line Interface
Ad hoc scripting and integration
Globus Flows service
Comprehensive task (data and
compute) orchestration with human in
the loop interactions

CLI v3
• Collection support
• Group support
• Search support
• Multiple profiles
• Python SDK updates
20

Globus Flows: Managed task automation
• Flows: A platform service for defining, applying, and
sharing distributed research automation flows
• Flows comprise Actions
• Action Providers: Called by Flows to perform tasks
• Triggers*: Start flows based on events
* In development

Serial Synchrotron Crystallography of SARS-CoV-2 proteins
www.alcf.anl.gov/news/argonne-researchers-use-theta-real-time-analysis-covid-19-proteins
The pipeline generates large image batches at a high rate, with data transfers achieving speeds of
700 megabytes per second thanks to Globus, a University of Chicago-run data management service.

Gladier: The Globus Architecture for Data-Intensive
Experimental Research
• Accelerate and simplify flow
development and deployment
• Combine tools into reliable,
flexible, secure, distributed
flows
• Bridge instruments and
computing facilities
• Automate data collection and
publication to create FAIR data

funcX: managed and federated FaaS
• Cloud-hosted service for managing compute
• Register and share compute endpoints
• Register and share Python functions
• Reliably, scalable, securely execute functions on
remote endpoints
• Integrated with Globus Auth and data ecosystem
25
Try funcx on Binder
https://funcx.org

Our Mission
Increase the efficiency and
effectiveness of researchers
engaged in data-driven
science and scholarship
through sustainable software

Weitere ähnliche Inhalte

Was ist angesagt?

Mario Molina, Software Engineer CDC systems are usually used to identify changes in data sources, capture and replicate those changes to other systems. Companies are using CDC to sync data across systems, cloud migration or even applying stream processing, among others. In this presentation we’ll see CDC patterns, how to use it in Apache Kafka, and do a live demo! https://www.meetup.com/Mexico-Kafka/events/277309497/

CDC patterns in Apache Kafka®

confluent

KSQL: Open Source Streaming for Apache Kafka

confluent

Agenda: Apache Kafka Ecosystem Kafka Streams as Foundation for KSQL Motivation for KSQL KSQL Concepts Live Demo #1 – Intro to KSQL KSQL Architecture Live Demo #2 - Clickstream Analysis Building a User Defined Function (Example: Machine Learning) Getting Started ### The rapidly expanding world of stream processing can be daunting, with new concepts such as various types of time semantics, windowed aggregates, changelogs, and programming frameworks to master. KSQL is an open-source, Apache 2.0 licensed streaming SQL engine on top of Apache Kafka which aims to simplify all this and make stream processing available to everyone. Even though it is simple to use, KSQL is built for mission-critical and scalable production deployments (using Kafka Streams under the hood). Benefits of using KSQL include No coding required; no additional analytics cluster needed; streams and tables as first-class constructs; access to the rich Kafka ecosystem. This session introduces the concepts and architecture of KSQL. Use cases such as Streaming ETL, Real-Time Stream Monitoring or Anomaly Detection are discussed. A live demo shows how to setup and use KSQL quickly and easily on top of your Kafka ecosystem.

KSQL Deep Dive - The Open Source Streaming Engine for Apache Kafka

Kai Wähner

Solving Enterprise Data Challenges with Apache Arrow

Wes McKinney

HTAP Queries

Atif Shaikh

Data-Streaming at DKV

confluent

Flink Forward San Francisco 2022. A particular feature in our system relies on a streaming 90-minute trailing window of 1-minute samples - implemented as a lookaside cache - to speed up a particular query, allowing our customers to rapidly see an overview of their estate. Across our entire customer base, there is a substantial amount of data flowing into this cache - ~1,000,000 entries/second, with the entire cache requiring ~600GB of RAM. The current implementation is simplistic but expensive. In this talk I describe a replacement implementation as a stateful streaming Flink application leveraging Queryable State. This Flink application reduces the net cost by ~90%. In this session, the implementation is described in detail, including windowing considerations, a sliding-window state buffer that avoids the sliding window replication penalty, and a comparison of queryable state and Redis queries. The talk concludes with a frank discussion of when this distinctive approach is, and is not, appropriate. by Ron Crocker

Using Queryable State for Fun and Profit

Flink Forward

Click-Through Example for Flink’s KafkaConsumer Checkpointing

Robert Metzger

Apple Siri is the world's largest virtual assistant service powering every iPhone, iPad, Mac, Apple TV, Apple Watch, and HomePod. We use large amounts of data to provide our users the best possible personalized experience. Our raw event data is cleaned and pre-joined into an unified data for our data consumers to use. To keep the rich hierarchical structure of the data, our data schemas are very deep nested structures. In this talk, we will discuss how Spark handles nested structures in Spark 2.4, and we'll show the fundamental design issues in reading nested fields which is not being well considered when Spark SQL was designed. This results in Spark SQL reading unnecessary data in many operations. Given that Siri's data is super nested and humongous, this soon becomes a bottleneck in our pipelines. Then we will talk about the various approaches we have taken to tackle this problem. By making nested columns as first citizen in Spark SQL, we can achieve dramatic performance gain. In some of our production queries, the speed-up can be 20x in wall clock time and 8x less data being read. All of our work will be open source, and some has already been merged into upstream.

Making Nested Columns as First Citizen in Apache Spark SQL

Databricks

Migrating to Globus Connect Server v5

Globus

Flink Forward San Francisco 2022. In modern data platform architectures, stream processing engines such as Apache Flink are used to ingest continuous streams of data into data lakes such as Apache Iceberg. Streaming ingestion to iceberg tables can suffer by two problems (1) small files problem that can hurt read performance (2) poor data clustering that can make file pruning less effective. To address those two problems, we propose adding a shuffling stage to the Flink Iceberg streaming writer. The shuffling stage can intelligently group data via bin packing or range partition. This can reduce the number of concurrent files that every task writes. It can also improve data clustering. In this talk, we will explain the motivations in details and dive into the design of the shuffling stage. We will also share the evaluation results that demonstrate the effectiveness of smart shuffling. by Gang Ye & Steven Wu

Tame the small files problem and optimize data layout for streaming ingestion...

Flink Forward

JupyterHub: Learning at Scale

Carol Willing

Search and recommendation system for Alibaba’s e-commerce platform use batch and streaming processing heavily. Flink SQL and Table API (which is a SQL-like DSL) provide simple, flexible, and powerful language to express the data processing logic. More importantly, it opens the door to unify the semantics of batch and streaming jobs. Blink is a project at Alibaba which improves Apache Flink to make it ready for large scale production use. To support our products, we made lots of improvements to Flink SQL & TableAPI in Alibaba's Blink project. We added the support for User-Defined Table function (UDTF), User-Defined Aggregates (UDAGG), Window Aggregate, and retraction, etc. We are actively working with the Flink community to contribute these improvements back. In this talk, we will present the rationale, semantics, design and implementation of these improvements. We will also share the experience of running large scale Flink SQL and TableAPI jobs at Alibaba.

Flink SQL & TableAPI in Large Scale Production at Alibaba

DataWorks Summit

Introducing Apache Airflow and how we are using it

Bruno Faria

Write Faster SQL with Trino.pdf

Eric Xiao

Flink Forward San Francisco 2022. When running Flink jobs, skew is a common problem that results in wasted resources and limited scalability. In the past years, we have helped our customers and users solve various skew-related issues in their Flink jobs or clusters. In this talk, we will present the different types of skew that users often run into: data skew, key skew, event time skew, state skew, and scheduling skew, and discuss solutions for each of them. We hope this will serve as a guideline to help you reduce skew in your Flink environment. by Jun Qin & Karl Friedrich

Evening out the uneven: dealing with skew in Flink

Flink Forward

Abstract: ReStore is an object-relational database interface for Dolphin Smalltalk. A key design philosophy behind ReStore is the use of familiar Smalltalk expressions in preference to relational database terminology. This is particularly evident in the querying subsystem which uses standard Collection expressions, for example: [[[ Employee storedInstances select: [ :each | each office city = ‘London’] ]]] Translation of the select block to a SQL query uses the familiar technique of evaluating the block with an analysis object, tracking each message send via doesNotUnderstand:. A shortcoming of this approach is the inability to handle commonly-used optimised/inlined expressions such as isNil, and: etc. This forces the use of less common or non-standard equivalents, compromising the design philosophy and leading to errors where code uses the more familiar expressions. This presentation describes improvements to ReStore’s query block analyser, using the reflective capabilities of Smalltalk to track optimised expressions via examination and manipulation of the execution stack. Bio: I have been working with Smalltalk since 1992, initially in academia and subsequently in a variety of employed and self-employed positions. From 2000 to 2005 I ran my own company creating bespoke systems for SMEs using Dolphin Smalltalk; this was the original impetus behind the creation of ReStore. Following ten years at JPMorgan working with VisualWorks and GemStone, in 2015 I relocated with my family to rural France and resumed working with Dolphin and ReStore.

Practical API-Development with Gemstone/S

ESUG

Dallas Mulesoft Meetup - Log Aggregation and Elastic Stack on Anypoint Platform

Adam DesJardin

Storage and Alfresco

Toni de la Fuente

Speed Up Uber's Presto with Alluxio

Alluxio, Inc.

Was ist angesagt? (20)

CDC patterns in Apache Kafka®

KSQL: Open Source Streaming for Apache Kafka

KSQL Deep Dive - The Open Source Streaming Engine for Apache Kafka

Solving Enterprise Data Challenges with Apache Arrow

HTAP Queries

Data-Streaming at DKV

Using Queryable State for Fun and Profit

Click-Through Example for Flink’s KafkaConsumer Checkpointing

Making Nested Columns as First Citizen in Apache Spark SQL

Migrating to Globus Connect Server v5

Tame the small files problem and optimize data layout for streaming ingestion...

JupyterHub: Learning at Scale

Flink SQL & TableAPI in Large Scale Production at Alibaba

Introducing Apache Airflow and how we are using it

Write Faster SQL with Trino.pdf

Evening out the uneven: dealing with skew in Flink

Practical API-Development with Gemstone/S

Dallas Mulesoft Meetup - Log Aggregation and Elastic Stack on Anypoint Platform

Storage and Alfresco

Speed Up Uber's Presto with Alluxio

Ähnlich wie What's New in Globus - Internet2 TechEXtra

GlobusWorld 2020 Keynote

Globus

Research computing facilities, such as the national supercomputing centers, and shared instruments, such as cryo electron microscopes and advanced light sources, are generating large volumes of data daily. These growing data volumes make it challenging for researchers to perform what should be mundane tasks: move data reliably, describe data for subsequent discovery, and make data accessible to geographically distributed collaborators. Most employ some set of ad hoc methods, which are not scalable, and it is clear that some level of automation is required for these tasks. Globus is an established service from the University of Chicago that is widely used for managing research data in national laboratories, campus computing centers, and HPC facilities. While its intuitive web app addresses simple file transfer and sharing scenarios, automation at scale requires integrating Globus data management platform services into custom science gateways, data portals and other web applications in service of research. Such applications should enable automated ingest of data from diverse sources, launching of analysis runs on diverse computing resources, extraction and addition of metadata for creating search indexes, assignment of persistent identifiers faceted search for rapid data discovery, and point-and-click downloading of datasets by authorized users — all protected by an authentication and authorization substrate that allows the implementation of flexible data access policies for both metadata and data alike. We describe current and emerging Globus services that facilitate these automated data flows while ensuring a streamlined user experience. We also demonstrate Petreldata.net, a data management portal and gateway to multiple computing resources, that supports large-scale research at the Advanced Photon Source.

Automating Research Data Management at Scale with Globus

Globus

Scientists have embraced the use of specialized cloud-hosted services to perform data management operations. Globus offers a suite of data and user management capabilities to the community, encompassing data transfer and sharing, user identity and authorization, and data publication. Globus capabilities are accessible via both a web browser and REST APIs. Web access allows Globus to address the needs of research labs through a software-as-a-service model; the newer REST APIs address the needs of developers of research services, who can now use Globus as a platform, outsourcing complex user and data management tasks to Globus cloud-hosted services. Here we review Globus capabilities and outline how it is being applied as a platform for scientific services. Presentation by Steve Tuecke from The University of Chicago. Steve is Globus Founder and Project Lead.

Globus: Research Data Management as Service and Platform - pearc17

Mary Bass

Simplified Research Data Management with the Globus Platform

Globus

Introduction to Globus for New Users

Globus

Federated Cloud Computing

David Wallom

Introduction to Globus for New Users

Globus

Ever more data- and compute-intensive science makes computing increasingly important for research. But for advanced computing infrastructure to benefit more than the scientific 1%, we need new delivery methods that slash access costs, new sustainability models beyond direct research funding, and new platform capabilities to accelerate the development of new, interoperable tools and services. The Globus team has been working towards these goals since 2010. We have developed software-as-a-service methods that move complex and time-consuming research IT tasks out of the lab and into the cloud, thus greatly reducing the expertise and resources required to use them. We have demonstrated a subscription-based funding model that engages research institutions in supporting service operations. And we are now also showing how the platform services that underpin Globus applications can accelerate the development and use of an integrated ecosystem of advanced science applications, such as NCAR’s Research Data Archive and OSG Connect, thus enabling access to powerful data and compute resources by many more people than is possible today. In this talk, I introduce Globus services and the underlying Globus platform. I present representative applications and discuss opportunities that this platform presents for both small science and large facilities.

Science Services and Science Platforms: Using the Cloud to Accelerate and Dem...

Ian Foster

Introduction to the Globus SaaS (GlobusWorld Tour - STFC)

Globus

Talk in the National Science Data Fabric (NSDF) Distinguished Speaker Series The Globus team has spent more than a decade developing software-as-a-service methods for research data management, available at globus.org. Globus transfer, sharing, search, publication, identity and access management (IAM), automation, and other services enable reliable, secure, and efficient managed access to exabytes of scientific data on tens of thousands of storage systems. For developers, flexible and open platform APIs reduce greatly the cost of developing and operating customized data distribution, sharing, and analysis applications. With 200,000 registered users at more than 2,000 institutions, more than 1.5 exabytes and 100 billion files handled, and 100s of registered applications and services, the services that comprise the Globus platform have become essential infrastructure for many researchers, projects, and institutions. I describe the design of the Globus platform, present illustrative applications, and discuss lessons learned for cyberinfrastructure software architecture, dissemination, and sustainability. Video is at https://www.youtube.com/watch?v=p8pCHkFFq1E

A Global Research Data Platform: How Globus Services Enable Scientific Discovery

Ian Foster

Globus: A Data Management Platform for Collaborative Research (CHPC 2019 - So...

Globus

"What's New With Globus" Webinar: Spring 2018

Globus

Introduction to Globus (APS Workshop)

Globus

Grid computing

Keshab Nath

Scaling Galaxy on Google Cloud Platform

Lynn Langit

Delivering petabyte-scale computational resources to a large community of users while meeting stringent security and compliance requirements presents a host of technical challenges. Seven Bridges Genomics met and overcame them when building the Cancer Genomics Cloud Pilot (CGC) for the National Cancer Institute. The CGC helps users to solve massive computational problems involving multidimensional data, which include: running diverse analyses in a reproducible manner, collaborating with other researchers, and keeping personal data secure to comply with NIH regulations on controlled data sets. Seven Bridges will highlight the lessons learned along the way, as well as best practices for constructing secure and compliant platform services using Amazon S3, Amazon Glacier, AWS Identity and Access Management (IAM), Amazon VPC, and Amazon Route 53.

(SEC313) Security & Compliance at the Petabyte Scale

Amazon Web Services

Experiences In Building Globus Genomics Using Galaxy, Globus Online and AWS

Ed Dodds

Introduction to Globus - XSEDE14 Tutorial

Globus

Globus is an established service from the University of Chicago that is widely used for managing research data in national laboratories, campus computing centers, and HPC facilities. While its interactive web browser interface addresses simple file transfer and sharing scenarios, large scale automation typically requires integration of the research data management platform it provides into bespoke applications. We will describe one such example, the Petrel data portal (https://petreldata.net), used by researchers to manage data in diverse fields including materials science, cosmology, machine learning, and serial crystallography. The portal facilitates automated ingest of data, extraction and addition of metadata for creating search indexes, assignment of persistent identifiers faceted search for rapid data discovery, and point-and-click downloading of datasets by authorized users. As security and privacy are often critical requirements, the portal employs fine-grained permissions that control both visibility of metadata and access to the datasets themselves. It is based on the Modern Research Data Portal design pattern, jointly developed by the ESnet and Globus teams, and leverages capabilities such as the Science DMZ for enhanced performance and to streamline the user experience.

Scalable Data Management: Automation and the Modern Research Data Portal

Globus

re:Invent 2013-foster-madduri

Ravi Madduri

Ähnlich wie What's New in Globus - Internet2 TechEXtra (20)

GlobusWorld 2020 Keynote

Automating Research Data Management at Scale with Globus

Globus: Research Data Management as Service and Platform - pearc17

Simplified Research Data Management with the Globus Platform

Introduction to Globus for New Users

Federated Cloud Computing

Introduction to Globus for New Users

Science Services and Science Platforms: Using the Cloud to Accelerate and Dem...

Introduction to the Globus SaaS (GlobusWorld Tour - STFC)

A Global Research Data Platform: How Globus Services Enable Scientific Discovery

Globus: A Data Management Platform for Collaborative Research (CHPC 2019 - So...

"What's New With Globus" Webinar: Spring 2018

Introduction to Globus (APS Workshop)

Grid computing

Scaling Galaxy on Google Cloud Platform

(SEC313) Security & Compliance at the Petabyte Scale

Experiences In Building Globus Genomics Using Galaxy, Globus Online and AWS

Introduction to Globus - XSEDE14 Tutorial

Scalable Data Management: Automation and the Modern Research Data Portal

re:Invent 2013-foster-madduri

Mehr von Globus

Advanced Globus System Administration Topics

Globus

We dive deeper into the Globus automation platform and describe how common instrument-based scenarios may be streamlined. We examine the various components of a Globus flow that takes data from the point of capture on an instrument through to distribution/publication of resulting products to collaborators. This material was presented at the Research Computing and Data Management Workshop, hosted by Rensselaer Polytechnic Institute on February 27-28, 2024.

Instrument Data Automation: The Life of a Flow

Globus

We provide a brief introduction to the Globus platform-as-a-service for developers, with emphasis on building simple web applications for data distribution and discovery. We describe how to register an application with Globus and access platform APIs using the Globus Python SDK and a Jupyter Notebook. We also introduce the Globus Search service and demonstrate how it is used by an open source web portal framework that can jumpstart research application development. This material was presented at the Research Computing and Data Management Workshop, hosted by Rensselaer Polytechnic Institute on February 27-28, 2024.

Building Research Applications with Globus PaaS

Globus

The function-as-a-service (FaaS) model is well established in commercial cloud offerings but less so in research computing environments. The Globus Compute service enables remote computing using the FaaS model, but allows users to execute functions on any compute resource where they have access. We provide an overview of the Globus Compute service, and demonstrate how to install an endpoint and execute a function on a remote system. This material was presented at the Research Computing and Data Management Workshop, hosted by Rensselaer Polytechnic Institute on February 27-28, 2024.

Reliable, Remote Computation at All Scales

Globus

Best Practices for Data Sharing Using Globus

Globus

We provide an overview of the Globus platform features, and demonstrate several data management features. Serving as an introductory session suitable for new users, we use the Globus web app to show data transfer and sharing, use of Globus Connect Personal for laptop/desktop access and introduce the Globus Command Line Interface for interactive and scripting. This material was presented at the Research Computing and Data Management Workshop, hosted by Rensselaer Polytechnic Institute on February 27-28, 2024.

An Introduction to Globus for Researchers

Globus

We present an overview of Globus services for automating research computing and data management tasks, to accelerate research process throughput. This content is aimed at researchers who wish to automate repetitive data management tasks (such as backup and data distribution to collaborators), as well as those working with instruments (cryoEM, next-gen sequencers, fMRI, etc.) who wish to streamline data egress, downstream analysis, and sharing at scale. This material was presented at the Research Computing and Data Management Workshop, hosted by Rensselaer Polytechnic Institute on February 27-28, 2024.

Introduction to Research Automation with Globus

Globus

We review the Globus Connect Server architecture and deployment model, and describe the process for creating a Globus endpoint on your HPC cluster, lab server, or other multi-user storage system. You will learn how to install Globus Connect Server, and configure a number of common options on the endpoint. This material was presented at the Research Computing and Data Management Workshop, hosted by Rensselaer Polytechnic Institute on February 27-28, 2024.

Globus for System Administrators

Globus

Introduction to Globus for System Administrators

Globus

Introduction to Data Transfer and Sharing for Researchers

Globus

Introduction to the Globus Platform for Developers

Globus

Introduction to the Command Line Interface (CLI)

Globus

Automating Research Data with Globus Flows and Compute

Globus

We introduce the various Globus approaches available for automating data flows, including the command line interface (CLI), the Globus Timer service and the Globus Flows service. We use simple CLI commands and a Jupyter notebook to automate file transfers and permissions management on shared datasets. We also provide a brief introduction to the Globus platform-as-a-service for developers, with emphasis on understanding the security model; and demonstrate how to access Globus services via APIs for integration with custom research applications.

Automating Research Data Flows and Introduction to the Globus Platform

Globus

Advanced Globus System Administration

Globus

Introduction to Globus for System Administrators

Globus

Working with Globus Platform Services and Portals

Globus

Globus Automation

Globus

Advanced Globus System Administration

Globus

Introduction to Globus

Globus

Mehr von Globus (20)

Advanced Globus System Administration Topics

Instrument Data Automation: The Life of a Flow

Building Research Applications with Globus PaaS

Reliable, Remote Computation at All Scales

Best Practices for Data Sharing Using Globus

An Introduction to Globus for Researchers

Introduction to Research Automation with Globus

Globus for System Administrators

Introduction to Globus for System Administrators

Introduction to Data Transfer and Sharing for Researchers

Introduction to the Globus Platform for Developers

Introduction to the Command Line Interface (CLI)

Automating Research Data with Globus Flows and Compute

Automating Research Data Flows and Introduction to the Globus Platform

Advanced Globus System Administration

Introduction to Globus for System Administrators

Working with Globus Platform Services and Portals

Globus Automation

Advanced Globus System Administration

Introduction to Globus

Kürzlich hochgeladen

The presentation explores the development and application of artificial intelligence (AI) from its inception to its current status in the modern world. The term "artificial intelligence" was first coined by John McCarthy in 1956 to describe efforts to develop computer programs capable of performing tasks that typically require human intelligence. This concept was first introduced at a conference held at Dartmouth College, where programs demonstrated capabilities such as playing chess, proving theorems, and interpreting texts. In the early stages, Alan Turing contributed to the field by defining intelligence as the ability of a being to respond to certain questions intelligently, proposing what is now known as the Turing Test to evaluate the presence of intelligent behavior in machines. As the decades progressed, AI evolved significantly. The 1980s focused on machine learning, teaching computers to learn from data, leading to the development of models that could improve their performance based on their experiences. The 1990s and 2000s saw further advances in algorithms and computational power, which allowed for more sophisticated data analysis techniques, including data mining. By the 2010s, the proliferation of big data and the refinement of deep learning techniques enabled AI to become mainstream. Notable milestones included the success of Google's AlphaGo and advancements in autonomous vehicles by companies like Tesla and Waymo. A major theme of the presentation is the application of generative AI, which has been used for tasks such as natural language text generation, translation, and question answering. Generative AI uses large datasets to train models that can then produce new, coherent pieces of text or other media. The presentation also discusses the ethical implications and the need for regulation in AI, highlighting issues such as privacy, bias, and the potential for misuse. These concerns have prompted calls for comprehensive regulations to ensure the safe and equitable use of AI technologies. Artificial intelligence has also played a significant role in healthcare, particularly highlighted during the COVID-19 pandemic, where it was used in drug discovery, vaccine development, and analyzing the spread of the virus. The capabilities of AI in healthcare are vast, ranging from medical diagnostics to personalized medicine, demonstrating the technology's potential to revolutionize fields beyond just technical or consumer applications. In conclusion, AI continues to be a rapidly evolving field with significant implications for various aspects of society. The development from theoretical concepts to real-world applications illustrates both the potential benefits and the challenges that come with integrating advanced technologies into everyday life. The ongoing discussion about AI ethics and regulation underscores the importance of managing these technologies responsibly to maximize their their benefits while minimizing potential harms.

Artificial Intelligence: Facts and Myths

Joaquim Jorge

Join our latest Connector Corner webinar to discover how UiPath Integration Service revolutionizes API-centric automation in a 'Quote to Cash' process—and how that automation empowers businesses to accelerate revenue generation. A comprehensive demo will explore connecting systems, GenAI, and people, through powerful pre-built connectors designed to speed process cycle times. Speakers: James Dickson, Senior Software Engineer Charlie Greenberg, Host, Product Marketing Manager

Connector Corner: Accelerate revenue generation using UiPath API-centric busi...

DianaGray10

Real Time Object Detection Using Open CV

Khem

AWS Community Day CPH - Three problems of Terraform

Andrey Devyatkin

2024: Domino Containers - The Next Step. News from the Domino Container commu...

Martijn de Jong

Tech Trends Report 2024 Future Today Institute.pdf

hans926745

💉💊+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHABI}}+971581248768 +971581248768 Mtp-Kit (500MG) Prices » Dubai [(+971581248768**)] Abortion Pills For Sale In Dubai, UAE, Mifepristone and Misoprostol Tablets Available In Dubai, UAE CONTACT DR.Maya Whatsapp +971581248768 We Have Abortion Pills / Cytotec Tablets /Mifegest Kit Available in Dubai, Sharjah, Abudhabi, Ajman, Alain, Fujairah, Ras Al Khaimah, Umm Al Quwain, UAE, Buy cytotec in Dubai +971581248768''''Abortion Pills near me DUBAI | ABU DHABI|UAE. Price of Misoprostol, Cytotec” +971581248768' Dr.DEEM ''BUY ABORTION PILLS MIFEGEST KIT, MISOPROTONE, CYTOTEC PILLS IN DUBAI, ABU DHABI,UAE'' Contact me now via What's App…… abortion Pills Cytotec also available Oman Qatar Doha Saudi Arabia Bahrain Above all, Cytotec Abortion Pills are Available In Dubai / UAE, you will be very happy to do abortion in Dubai we are providing cytotec 200mg abortion pill in Dubai, UAE. Medication abortion offers an alternative to Surgical Abortion for women in the early weeks of pregnancy. We only offer abortion pills from 1 week-6 Months. We then advise you to use surgery if its beyond 6 months. Our Abu Dhabi, Ajman, Al Ain, Dubai, Fujairah, Ras Al Khaimah (RAK), Sharjah, Umm Al Quwain (UAQ) United Arab Emirates Abortion Clinic provides the safest and most advanced techniques for providing non-surgical, medical and surgical abortion methods for early through late second trimester, including the Abortion By Pill Procedure (RU 486, Mifeprex, Mifepristone, early options French Abortion Pill), Tamoxifen, Methotrexate and Cytotec (Misoprostol). The Abu Dhabi, United Arab Emirates Abortion Clinic performs Same Day Abortion Procedure using medications that are taken on the first day of the office visit and will cause the abortion to occur generally within 4 to 6 hours (as early as 30 minutes) for patients who are 3 to 12 weeks pregnant. When Mifepristone and Misoprostol are used, 50% of patients complete in 4 to 6 hours; 75% to 80% in 12 hours; and 90% in 24 hours. We use a regimen that allows for completion without the need for surgery 99% of the time. All advanced second trimester and late term pregnancies at our Tampa clinic (17 to 24 weeks or greater) can be completed within 24 hours or less 99% of the time without the need surgery. The procedure is completed with minimal to no complications. Our Women's Health Center located in Abu Dhabi, United Arab Emirates, uses the latest medications for medical abortions (RU-486, Mifeprex, Mifegyne, Mifepristone, early options French abortion pill), Methotrexate and Cytotec (Misoprostol). The safety standards of our Abu Dhabi, United Arab Emirates Abortion Doctors remain unparalleled. They consistently maintain the lowest complication rates throughout the nation. Our Physicians and staff are always available to answer questions and care for women in one of the most difficult times in their lives. The decision to have an abortion at the Abortion Cl

+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...

?#DUbAI#??##{{(☎️+971_581248768%)**%*]'#abortion pills for sale in dubai@

GenCyber Cyber Security Day Presentation

Michael W. Hawkins

GenAI Risks & Security Meetup 01052024.pdf

lior mazor

Partners Life - Insurer Innovation Award 2024

The Digital Insurer

Handwritten Text Recognition for manuscripts and early printed texts

Maria Levchenko

Discord is a free app offering voice, video, and text chat functionalities, primarily catering to the gaming community. It serves as a hub for users to create and join servers tailored to their interests. Discord’s ecosystem comprises servers, each functioning as a distinct online community with its own channels dedicated to specific topics or activities. Users can engage in text-based discussions, voice calls, or video chats within these channels. Understanding Discord Servers Discord servers are virtual spaces where users congregate to interact, share content, and build communities. Servers may revolve around gaming, hobbies, interests, or fandoms, providing a platform for like-minded individuals to connect. Communication Features Discord offers a range of communication tools, including text channels for messaging, voice channels for real-time audio conversations, and video channels for face-to-face interactions. These features facilitate seamless communication and collaboration. What Does NSFW Mean? The acronym NSFW stands for “Not Safe For Work,” indicating content that may be inappropriate for professional or public settings. NSFW Content NSFW content encompasses material that is sexually explicit, violent, or otherwise graphic in nature. It often includes nudity, profanity, or depictions of sensitive topics.

Understanding Discord NSFW Servers A Guide for Responsible Users.pdf

UK Journal

Created by Mozilla Research in 2012 and now part of Linux Foundation Europe, the Servo project is an experimental rendering engine written in Rust. It combines memory safety and concurrency to create an independent, modular, and embeddable rendering engine that adheres to web standards. Stewardship of Servo moved from Mozilla Research to the Linux Foundation in 2020, where its mission remains unchanged. After some slow years, in 2023 there has been renewed activity on the project, with a roadmap now focused on improving the engine’s CSS 2 conformance, exploring Android support, and making Servo a practical embeddable rendering engine. In this presentation, Rakhi Sharma reviews the status of the project, our recent developments in 2023, our collaboration with Tauri to make Servo an easy-to-use embeddable rendering engine, and our plans for the future to make Servo an alternative web rendering engine for the embedded devices industry. (c) Embedded Open Source Summit 2024 April 16-18, 2024 Seattle, Washington (US) https://events.linuxfoundation.org/embedded-open-source-summit/ https://ossna2024.sched.com/event/1aBNF/a-year-of-servo-reboot-where-are-we-now-rakhi-sharma-igalia

A Year of the Servo Reboot: Where Are We Now?

Igalia

With more memory available, system performance of three Dell devices increased, which can translate to a better user experience Conclusion When your system has plenty of RAM to meet your needs, you can efficiently access the applications and data you need to finish projects and to-do lists without sacrificing time and focus. Our test results show that with more memory available, three Dell PCs delivered better performance and took less time to complete the Procyon Office Productivity benchmark. These advantages translate to users being able to complete workflows more quickly and multitask more easily. Whether you need the mobility of the Latitude 5440, the creative capabilities of the Precision 3470, or the high performance of the OptiPlex Tower Plus 7010, configuring your system with more RAM can help keep processes running smoothly, enabling you to do more without compromising performance.

Boost PC performance: How more available memory can improve productivity

Principled Technologies

Developing An App To Navigate The Roads of Brazil

V3cube

Abhishek Deb(1), Mr Abdul Kalam(2) M. Des (UX) , School of Design, DIT University , Dehradun. This paper explores the future potential of AI-enabled smartphone processors, aiming to investigate the advancements, capabilities, and implications of integrating artificial intelligence (AI) into smartphone technology. The research study goals consist of evaluating the development of AI in mobile phone processors, analyzing the existing state as well as abilities of AI-enabled cpus determining future patterns as well as chances together with reviewing obstacles as well as factors to consider for more growth.

Exploring the Future Potential of AI-Enabled Smartphone Processors

debabhi2

Advantages of Hiring UIUX Design Service Providers for Your Business

Pixlogix Infotech

Imagine a world where information flows as swiftly as thought itself, making decision-making as fluid as the data driving it. Every moment is critical, and the right tools can significantly boost your organization’s performance. The power of real-time data automation through FME can turn this vision into reality. Aimed at professionals eager to leverage real-time data for enhanced decision-making and efficiency, this webinar will cover the essentials of real-time data and its significance. We’ll explore: FME’s role in real-time event processing, from data intake and analysis to transformation and reporting An overview of leveraging streams vs. automations FME’s impact across various industries highlighted by real-life case studies Live demonstrations on setting up FME workflows for real-time data Practical advice on getting started, best practices, and tips for effective implementation Join us to enhance your skills in real-time data automation with FME, and take your operational capabilities to the next level.

From Event to Action: Accelerate Your Decision Making with Real-Time Automation

Safe Software

MySQL Webinar, presented on the 25th of April, 2024. Summary: MySQL solutions enable the deployment of diverse Database Architectures tailored to specific needs, including High Availability, Disaster Recovery, and Read Scale-Out. With MySQL Shell's AdminAPI, administrators can seamlessly set up, manage, and monitor these solutions, ensuring efficiency and ease of use in their administration. MySQL Router, on the other hand, provides transparent routing from the application traffic to the backend servers in the architectures, requiring minimal configuration. Completely built in-house and supported by Oracle, these solutions have been adopted by enterprises of all sizes for their business-critical applications. In this presentation, we'll delve into various database architecture solutions to help you choose the right one based on your business requirements. Focusing on technical details and the latest features to maximize the potential of these solutions.

Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...

Miguel Araújo

In this session, we will delve into strategic approaches for optimizing knowledge management within Microsoft 365, amidst the evolving landscape of Copilot. From leveraging automatic metadata classification and permission governance with SharePoint Premium, to unlocking Viva Engage for the cultivation of knowledge and communities, you will gain actionable insights to bolster your organization's knowledge-sharing initiatives. In this session, we will also explore how to facilitate solutions to enable your employees to find answers and expertise within Microsoft 365. You will leave equipped with practical techniques and a deeper understanding of how there is more to effective knowledge management than just enabling Copilot, but building actual solutions to prepare the knowledge that Copilot and your employees can use.

Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...

Drew Madelung

Kürzlich hochgeladen (20)

Artificial Intelligence: Facts and Myths

Connector Corner: Accelerate revenue generation using UiPath API-centric busi...

Real Time Object Detection Using Open CV

AWS Community Day CPH - Three problems of Terraform

2024: Domino Containers - The Next Step. News from the Domino Container commu...

Tech Trends Report 2024 Future Today Institute.pdf

+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...

GenCyber Cyber Security Day Presentation

GenAI Risks & Security Meetup 01052024.pdf

Partners Life - Insurer Innovation Award 2024

Handwritten Text Recognition for manuscripts and early printed texts

Understanding Discord NSFW Servers A Guide for Responsible Users.pdf

A Year of the Servo Reboot: Where Are We Now?

Boost PC performance: How more available memory can improve productivity

Developing An App To Navigate The Roads of Brazil

Exploring the Future Potential of AI-Enabled Smartphone Processors

Advantages of Hiring UIUX Design Service Providers for Your Business

From Event to Action: Accelerate Your Decision Making with Real-Time Automation

Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...

Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...

What's New in Globus - Internet2 TechEXtra

1. What’s New in Globus Vas Vasiliadis – vas@uchicago.edu Internet2 TechEXtra – February 23, 2022

2. It’s been a long while… 2

3. What a difference three years make! We’ve been busy… • …Rebuilding the technical foundation • …Growing the connected storage ecosystem • …Expanding the data management platform • …Automating data management at scale 3

4. Rebuilding the technical foundation: GCSv5 • Modern security model • Support compliance requirements • Deployment flexibility • Enhanced sharing policies • Simplified scaling/availability 4

5. GCSv5 Architecture 5

6. Growing the connected storage ecosystem

7. Data-appropriate storage Google Drive for project admin files Google Cloud Storage for core research files Reliably manage API constraints

8. Support for all major cloud storage providers

9. Simplifying cloud storage migration 9

10. Partnership with the community to develop new connectors Community Connector Program

11. Easy egress and ingress of data Data sharing with collaborators via unified interface

12. POSIX Staging Connector • Example: – IBM Spectrum Archive plugin, Brock Palen at University of Michigan - github.com/brockpalen/ltfsee-globus 12 • For POSIX file systems that cache from tertiary storage • Custom plug-in for staging files

13. Streamlined GCP Installation • Log in from the application during installation • Use setup key method for automation

14. New platform services • Groups • Search • Flows 14

15. Globus Groups docs.globus.org/api/groups groups.api.globus.org/redoc 15

16. Globus Search: Data description/discovery • Metadata store with fine- grained visibility controls • Schema agnostic à dynamic schemas • Simple search using URL query parameters • Complex search using search request document • Example: 16 docs.globus.org/api/search Search Index github.com/globus/searchable-files-demo

17. Globus Search Terra Fusion on OSN 17 osn.globusdemo.org

18. 3 Degrees of Automation Timer Service Scheduled and recurring transfers (a.k.a. Globus cron) Command Line Interface Ad hoc scripting and integration Globus Flows service Comprehensive task (data and compute) orchestration with human in the loop interactions

19. Globus Timer Service 19

20. CLI v3 • Collection support • Group support • Search support • Multiple profiles • Python SDK updates 20

21. Globus Flows: Managed task automation • Flows: A platform service for defining, applying, and sharing distributed research automation flows • Flows comprise Actions • Action Providers: Called by Flows to perform tasks • Triggers*: Start flows based on events * In development

22. Serial Synchrotron Crystallography of SARS-CoV-2 proteins www.alcf.anl.gov/news/argonne-researchers-use-theta-real-time-analysis-covid-19-proteins The pipeline generates large image batches at a high rate, with data transfers achieving speeds of 700 megabytes per second thanks to Globus, a University of Chicago-run data management service.

23. Gladier: The Globus Architecture for Data-Intensive Experimental Research • Accelerate and simplify flow development and deployment • Combine tools into reliable, flexible, secure, distributed flows • Bridge instruments and computing facilities • Automate data collection and publication to create FAIR data

24. 24

25. funcX: managed and federated FaaS • Cloud-hosted service for managing compute • Register and share compute endpoints • Register and share Python functions • Reliably, scalable, securely execute functions on remote endpoints • Integrated with Globus Auth and data ecosystem 25 Try funcx on Binder https://funcx.org

26. Our Mission Increase the efficiency and effectiveness of researchers engaged in data-driven science and scholarship through sustainable software

What's New in Globus - Internet2 TechEXtra

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Ähnlich wie What's New in Globus - Internet2 TechEXtra

Ähnlich wie What's New in Globus - Internet2 TechEXtra (20)

Mehr von Globus

Mehr von Globus (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

What's New in Globus - Internet2 TechEXtra