Three-dimensional Particle Tracking Velocimetry is the state-of-the-art fluid flow measurement method, providing a unique view into Lagrangian statistics of a turbulent flow in the urban canopy wind tunnel model.
This is the first poster I presented as part of my PhD. It focuses on executing N-body simulations using GRAPE specialized hardware on machines in different continents.
This document presents a case study running high-performance computing (HPC) applications in public clouds. It evaluates the performance and economic feasibility of running benchmarks and a full-size NASA application across Amazon EC2, GoGrid, and IBM cloud infrastructures. The study finds that while HPC applications can run in the clouds, performance often lags dedicated HPC systems and scales poorly with problem size. Embarrassingly parallel applications achieve the best results. Network performance and economics remain challenges for using public clouds for HPC.
Sante presents ideas they will work on in their new position conducting multidisciplinary studies connecting general relativity and other fields. They discuss IDSS RPS, a generalization of GPS that accounts for relativistic effects without needing continuous broadcasting, prior knowledge of trajectories, or geodesic tracing. They also discuss using metamaterials and analogue transformations to design acoustic devices by mapping acoustic waves in media to relativistic systems, as well as potential applications in other fields like chemistry and aerodynamics. Sante invites those interested to contact them to discuss other related ideas.
Improving Passive Packet Capture : Beyond Device PollingHargyo T. Nugroho
The document discusses improving passive packet capture performance beyond device polling. It proposes a "Socket Ring" approach using PF_RING to create a ring buffer on the network interface card driver. This allows captured packets to bypass the kernel and be directly accessed by userspace applications via memory mapping, improving performance over traditional approaches. Experimental results found the PF_RING approach captured packets much faster than Linux's standard approach, especially for medium and large packets, though some packets were still lost. The approach requires a real-time kernel patch and performance is ultimately limited by network drivers and how the kernel fetches packets.
Cloud computing provides outsourced computing infrastructure and tools like Hadoop and Dryad for data-parallel processing. Commercial clouds are proprietary but open-source versions exist. Building open-architecture clouds requires understanding hardware, virtualization, services, and runtimes best practices. Cloud runtimes can run data-file parallel and dataflow applications at large scales for problems in areas like biology, geospatial processing, and clustering. Deterministic annealing is a parallelizable algorithm for data clustering that has been run on clouds. Clouds may change scientific computing by providing controllable, sustainable infrastructure without local clusters.
Surveillance scene classification using machine learningUtkarsh Contractor
The problem of scene classification in surveillance footage is of great importance for ensuring security in public areas. With challenges such as low quality feeds, occlusion, viewpoint variations, background clutter etc. The task is both challenging and error-prone. Therefore it is important to keep the false positives low to maintain a high accuracy of detection. In this paper, we adapt high performing CNN architectures to identify abandoned luggage in a surveillance feed. We explore several CNN based approaches, from Transfer Learning on the Imagenet dataset to object classification using Faster R-CNNs on the COCO dataset. Using network visualization techniques, we gain insight into what the neural network sees and the basis of classification decision. The experiments have been conducted on real world datasets, and highlights the complexity in such classifications. Obtained results indicate that a combination of proposed techniques outperforms the individual approaches.
Artificial Neural Networks for Storm Surge Prediction in North CarolinaAnton Bezuglov
Feedforward Artificial Neural network (FF ANN) for storm surge prediction in North Carolina. Presentation at Coastal Resilience Center by Anton Bezuglov, Ph.D. Usage of TensorFlow and Python with links to the code on GitHub.
This is the first poster I presented as part of my PhD. It focuses on executing N-body simulations using GRAPE specialized hardware on machines in different continents.
This document presents a case study running high-performance computing (HPC) applications in public clouds. It evaluates the performance and economic feasibility of running benchmarks and a full-size NASA application across Amazon EC2, GoGrid, and IBM cloud infrastructures. The study finds that while HPC applications can run in the clouds, performance often lags dedicated HPC systems and scales poorly with problem size. Embarrassingly parallel applications achieve the best results. Network performance and economics remain challenges for using public clouds for HPC.
Sante presents ideas they will work on in their new position conducting multidisciplinary studies connecting general relativity and other fields. They discuss IDSS RPS, a generalization of GPS that accounts for relativistic effects without needing continuous broadcasting, prior knowledge of trajectories, or geodesic tracing. They also discuss using metamaterials and analogue transformations to design acoustic devices by mapping acoustic waves in media to relativistic systems, as well as potential applications in other fields like chemistry and aerodynamics. Sante invites those interested to contact them to discuss other related ideas.
Improving Passive Packet Capture : Beyond Device PollingHargyo T. Nugroho
The document discusses improving passive packet capture performance beyond device polling. It proposes a "Socket Ring" approach using PF_RING to create a ring buffer on the network interface card driver. This allows captured packets to bypass the kernel and be directly accessed by userspace applications via memory mapping, improving performance over traditional approaches. Experimental results found the PF_RING approach captured packets much faster than Linux's standard approach, especially for medium and large packets, though some packets were still lost. The approach requires a real-time kernel patch and performance is ultimately limited by network drivers and how the kernel fetches packets.
Cloud computing provides outsourced computing infrastructure and tools like Hadoop and Dryad for data-parallel processing. Commercial clouds are proprietary but open-source versions exist. Building open-architecture clouds requires understanding hardware, virtualization, services, and runtimes best practices. Cloud runtimes can run data-file parallel and dataflow applications at large scales for problems in areas like biology, geospatial processing, and clustering. Deterministic annealing is a parallelizable algorithm for data clustering that has been run on clouds. Clouds may change scientific computing by providing controllable, sustainable infrastructure without local clusters.
Surveillance scene classification using machine learningUtkarsh Contractor
The problem of scene classification in surveillance footage is of great importance for ensuring security in public areas. With challenges such as low quality feeds, occlusion, viewpoint variations, background clutter etc. The task is both challenging and error-prone. Therefore it is important to keep the false positives low to maintain a high accuracy of detection. In this paper, we adapt high performing CNN architectures to identify abandoned luggage in a surveillance feed. We explore several CNN based approaches, from Transfer Learning on the Imagenet dataset to object classification using Faster R-CNNs on the COCO dataset. Using network visualization techniques, we gain insight into what the neural network sees and the basis of classification decision. The experiments have been conducted on real world datasets, and highlights the complexity in such classifications. Obtained results indicate that a combination of proposed techniques outperforms the individual approaches.
Artificial Neural Networks for Storm Surge Prediction in North CarolinaAnton Bezuglov
Feedforward Artificial Neural network (FF ANN) for storm surge prediction in North Carolina. Presentation at Coastal Resilience Center by Anton Bezuglov, Ph.D. Usage of TensorFlow and Python with links to the code on GitHub.
1. Space, Time, Power: Evolving Concerns for Parallel Algorithms February 2008
2. Real and Abstract Parallel Systems • Space: where are the processors located? • Time: how does location affect the time of algorithms? • Power: what happens when power is a constraint?
3. Some Real Systems: IBM BlueGene/L 212,992 CPUs 478 Tflops #1 supercomputer since 11/04 At Lawrence Livermore Nat’l Lab ≈ $200 Million 3-d toroidal interconnect Max distance (# proc)1/3
4. Another Real System: ZebraNet PI M M a r t o n o s i
5. Location, Location, Location • Processors may only be able to communicate with nearby processors • or, time to communicate is a function of distance • or, many processors trying to communicate to ones far away can create communication bottleneck • Feasible, efficient programs need to take location into account
6. What if Space is actually Computers? Cellular Automata • Finite automata, next state depends on current state and neighbors’ states: location matters! • ≈ 1950 von Neumann used as a model of parallelism and interaction in space • Other research: Burks & al. at UM, Conway, Wolfram,… • Can model leaf growth, traffic flow, etc.
7. Parallel Algorithms: Time Maze of black/white pixels, one per processor in CA. Can I get out? Nature-like propagation algorithm: time linear in area Beyer, Levialdi ≈ 1970: time linear in edgelength. CA as parallel computer, not just nature simulator
Project Matsu: Elastic Clouds for Disaster ReliefRobert Grossman
The document discusses Project Matsu, an initiative by the Open Cloud Consortium to provide cloud computing resources for large-scale image processing to assist with disaster relief. It proposes three technical approaches: 1) Using Hadoop and MapReduce to process images in parallel across nodes; 2) Using Hadoop streaming with Python to preprocess images into a single file for processing; and 3) Using the Sector distributed file system and Sphere UDFs to process images while keeping them together on nodes without splitting files. The overall goal is to enable elastic computing on petabyte-scale image datasets for change detection and other analyses to support disaster response.
A quantum computer harnesses quantum phenomena like quantum interference to process information in a fundamentally new way compared to classical computers. In a quantum computer, the basic unit of information called a qubit is quaternary rather than binary in nature. Experiments show that quantum detectors register particles 100% of the time at one detector and never at the other, which is puzzling. The premier application of a capable quantum computer is cracking encryption codes like RSA that rely on the difficulty of factoring large numbers, an operation best suited for quantum computers. However, building quantum computers faces formidable challenges including error correction, decoherence, and hardware architecture.
At the Knime Berlin summit 2016, Prof. Dr. Dominique Genoud presented a novel way to implement a KNIME workflow that perform machine learning and signal processing on an Android platform. The use case was to detect soft falls (not from a standing position) using an Android watch. This application has a big impact on how we can detect automatically when elderly people fall from their bed of their chair. This work was originally based on the Master Thesis in Business Administration realized by Vincent Cuendet in 2015 at the HES-SO with the help of the FST (Fédération Suisse pour les Téléthèses), an organization that helps disabled and elderly people to keep their autonomy.
Sky Arrays - ArrayDB in action for Sky View Factor ComputationEUDAT
1) ArrayDB was used to compute sky view factors over the entire Netherlands from light detection and ranging (LIDAR) data to analyze factors like heat stress and fog formation.
2) The previous method of processing large point cloud data files was memory intensive, required tracking file locations, and had high memory requirements. ArrayDB provided a standardized web-based interface.
3) The new workflow involved pre-processing LIDAR data into rasters, ingesting them into ArrayDB, then querying subsets for distributed computation of sky view factors in R instead of processing entire files. This made interaction easier and less error-prone.
This document describes several object detection methods: R-CNN, SPP-Net, Fast R-CNN, and a unified approach. R-CNN was slow due to processing each region of interest separately. SPP-Net addressed this with spatial pyramid pooling to process regions together. Fast R-CNN further improved speed by jointly training the detector. A unified approach aims to detect objects without extracting regions of interest by dividing the image into grids and predicting objects within each grid.
1) The document discusses using high-performance distributed computing to solve large-scale multimedia problems like analyzing vast amounts of video footage.
2) It describes Parallel-Horus, a tool that makes parallel and distributed computing transparent to users through sequential programming patterns and lazy parallelization.
3) Experiments show Parallel-Horus can achieve real-time performance for tasks like color-based object recognition by distributing the work across multiple computer clusters.
Neutron noise-based core monitoring techniques are being developed to detect anomalies in operating nuclear reactors. The CORTEX project aims to advance these techniques by developing simulation tools, validating them with experiments, and applying advanced signal processing methods. This will help characterize anomalies and their root causes to improve reactor monitoring and safety. The CORTEX consortium includes experts from 17 European organizations and will work towards demonstrating these techniques using data from operating plants.
This document proposes algorithms for floorplanning reconfigurable functional units on reconfigurable architectures. It aims to minimize fragmentation while considering device heterogeneity, reconfiguration capabilities, and inter-unit communication. The approach partitions a task graph into reconfigurable regions, floorplans units within each region, and then floorplans the regions on the device using simulated annealing. Experimental results show improvements over existing approaches in area usage and communication performance.
With the HPC Cloud facility, SURFsara offers self-service, dynamically scalable and fully configurable HPC systems to the Dutch academic community. Users have, for example, a free choice of operating system and software.
The HPC Cloud offers full control over a HPC cluster, with fast CPUs and high memory nodes and it is possible to attach terabytes of local storage to a compute node. Because of this flexibility, users can fully tailor the system for a particular application. Long-running and small compute jobs are equally welcome. Additionally, the system facilitates collaboration: users can share control over their virtual private HPC cluster with other users and share processing time, data and results. A portal with wiki, fora, repositories, issue system, etc. is offered for collaboration projects as well.
Real-Time Analysis of Streaming Synchotron Data: SCinet SC19 Technology Chall...Globus
This document describes a real-time workflow for analyzing streaming synchrotron data using high-performance computing resources. Synchrotron experiments produce large amounts of data that need to be reconstructed in real-time. The workflow includes data acquisition from multiple simulated beamlines, distribution to compute nodes, tomographic reconstruction using TraceX, denoising using TomoGAN, and visualization of results. A demonstration of this workflow is being run on Argonne Leadership Computing Facility's Theta supercomputer to process streaming data from 16,000 cores in real-time and provide reconstructed volumes and feedback to experiments.
The document discusses the National Research Platform (NRP), specifically the 4th NRP workshop. It provides an overview of NRP's Nautilus, a multi-institution hypercluster connected by optical networks across 25 partner campuses. In 2022, Nautilus comprised ~200 computing nodes and 4000TB of rotating storage. The document highlights several large research projects from different domains that utilize Nautilus, including particle physics, telescopes, biomedical applications, earth sciences, and visualization. These applications demonstrate how Nautilus enables data-intensive and collaborative multi-campus research at national scale.
How to build a TraP: An image-plane transient-discovery toolTim Staley
There are three main points summarized:
1. There are many interesting slow radio transients that could be detected through imaging surveys like accretion flares, orphan gamma-ray bursts, and flare stars.
2. Radio surveys are increasing in sensitivity and field of view by orders of magnitude with instruments like LOFAR, enabling the detection of more rare transient events.
3. TraP is a transient detection pipeline that works by extracting sources from radio images, matching to known sources, identifying new bright sources, analyzing light curves, and making the results accessible through a user-friendly web interface.
This document summarizes a research paper that developed a GPU-based algorithm for CT image reconstruction from undersampled and noisy projection data. The algorithm uses an algebraic reconstruction method and implements the LSQR iterative solver on a GPU using CUDA programming. By taking advantage of massive parallel processing on the GPU, the algorithm is able to reconstruct CT images with higher resolution than previous methods without losing image quality or increasing computation time significantly. The paper presents the mathematical model, reconstruction algorithm steps, and GPU implementation details.
Evaluating Classification Algorithms Applied To Data Streams Esteban DonatoEsteban Donato
This document summarizes and evaluates several algorithms for classification of data streams: VFDTc, UFFT, and CVFDT. It describes their approaches for handling concept drift, detecting outliers and noise. The algorithms were tested on synthetic data streams generated with configurable attributes like drift frequency and noise percentage. Results show VFDTc and UFFT performed best in accuracy, while CVFDT and UFFT were fastest. The study aims to help choose algorithms suitable for different data stream characteristics like gradual vs sudden drift or frequent vs infrequent drift.
Toward Real-Time Analysis of Large Data Volumes for Diffraction Studies by Ma...EarthCube
Talk at the EarthCube End-User Domain Workshop for Rock Deformation and Mineral Physics Research.
By Martin Kunz, Lawrence Berkeley National Laboratory
Real-Time Pedestrian Detection Using Apache Storm in a Distributed Environment csandit
This document discusses real-time pedestrian detection using Apache Storm in a distributed environment. It proposes a methodology that distributes video frames across multiple nodes ("bolts") for parallel processing. Each bolt detects pedestrians in a different region of the frame. Only detection results, not full frames, are transmitted to reduce overhead. The methodology is implemented using Apache Storm and evaluated on a test system with improved processing speed. Pedestrian detection has applications in safety and analytics.
REAL-TIME PEDESTRIAN DETECTION USING APACHE STORM IN A DISTRIBUTED ENVIRONMENTcscpconf
In general, a distributed processing is not suitable for dealing with image data stream due to the
network load problem caused by communications of frames. For this reason, image data stream
processing has operated in just one node commonly. However, we need to process image data
stream in a distributed environment in a big data era due to increase in quantity and quality of
multimedia data. In this paper, we shall present a real-time pedestrian detection methodology in
a distributed environment which processes image data stream in real-time on Apache Storm
framework. It achieves sharp speed up by distributing frames onto several nodes called bolts,
each of which processes different regions of image frames. Moreover, it can reduce the
overhead caused by synchronization by computation bolts which returns only the processing
results to the merging bolts.
This document provides information on several remote sensing projects from IEEE 2015. It lists the titles, languages, and abstracts for 8 projects related to classification and analysis of hyperspectral and multispectral images. The projects focus on techniques such as sparse representation in tangent space, Gabor feature-based collaborative representation, level set evolutions for object extraction, and dimension reduction using spatial and spectral regularization.
This document describes algorithms for detecting single radio pulses in real-time using graphics processing units (GPUs). It presents two new algorithms that use incomplete sets of boxcar filters to detect pulses at accelerated speeds with minimal signal loss. The algorithms were tested on simulated data and were found to process data 266-500 times faster than real-time on GPUs, detecting pulses with a mean 7% reduction in signal power.
1. Space, Time, Power: Evolving Concerns for Parallel Algorithms February 2008
2. Real and Abstract Parallel Systems • Space: where are the processors located? • Time: how does location affect the time of algorithms? • Power: what happens when power is a constraint?
3. Some Real Systems: IBM BlueGene/L 212,992 CPUs 478 Tflops #1 supercomputer since 11/04 At Lawrence Livermore Nat’l Lab ≈ $200 Million 3-d toroidal interconnect Max distance (# proc)1/3
4. Another Real System: ZebraNet PI M M a r t o n o s i
5. Location, Location, Location • Processors may only be able to communicate with nearby processors • or, time to communicate is a function of distance • or, many processors trying to communicate to ones far away can create communication bottleneck • Feasible, efficient programs need to take location into account
6. What if Space is actually Computers? Cellular Automata • Finite automata, next state depends on current state and neighbors’ states: location matters! • ≈ 1950 von Neumann used as a model of parallelism and interaction in space • Other research: Burks & al. at UM, Conway, Wolfram,… • Can model leaf growth, traffic flow, etc.
7. Parallel Algorithms: Time Maze of black/white pixels, one per processor in CA. Can I get out? Nature-like propagation algorithm: time linear in area Beyer, Levialdi ≈ 1970: time linear in edgelength. CA as parallel computer, not just nature simulator
Project Matsu: Elastic Clouds for Disaster ReliefRobert Grossman
The document discusses Project Matsu, an initiative by the Open Cloud Consortium to provide cloud computing resources for large-scale image processing to assist with disaster relief. It proposes three technical approaches: 1) Using Hadoop and MapReduce to process images in parallel across nodes; 2) Using Hadoop streaming with Python to preprocess images into a single file for processing; and 3) Using the Sector distributed file system and Sphere UDFs to process images while keeping them together on nodes without splitting files. The overall goal is to enable elastic computing on petabyte-scale image datasets for change detection and other analyses to support disaster response.
A quantum computer harnesses quantum phenomena like quantum interference to process information in a fundamentally new way compared to classical computers. In a quantum computer, the basic unit of information called a qubit is quaternary rather than binary in nature. Experiments show that quantum detectors register particles 100% of the time at one detector and never at the other, which is puzzling. The premier application of a capable quantum computer is cracking encryption codes like RSA that rely on the difficulty of factoring large numbers, an operation best suited for quantum computers. However, building quantum computers faces formidable challenges including error correction, decoherence, and hardware architecture.
At the Knime Berlin summit 2016, Prof. Dr. Dominique Genoud presented a novel way to implement a KNIME workflow that perform machine learning and signal processing on an Android platform. The use case was to detect soft falls (not from a standing position) using an Android watch. This application has a big impact on how we can detect automatically when elderly people fall from their bed of their chair. This work was originally based on the Master Thesis in Business Administration realized by Vincent Cuendet in 2015 at the HES-SO with the help of the FST (Fédération Suisse pour les Téléthèses), an organization that helps disabled and elderly people to keep their autonomy.
Sky Arrays - ArrayDB in action for Sky View Factor ComputationEUDAT
1) ArrayDB was used to compute sky view factors over the entire Netherlands from light detection and ranging (LIDAR) data to analyze factors like heat stress and fog formation.
2) The previous method of processing large point cloud data files was memory intensive, required tracking file locations, and had high memory requirements. ArrayDB provided a standardized web-based interface.
3) The new workflow involved pre-processing LIDAR data into rasters, ingesting them into ArrayDB, then querying subsets for distributed computation of sky view factors in R instead of processing entire files. This made interaction easier and less error-prone.
This document describes several object detection methods: R-CNN, SPP-Net, Fast R-CNN, and a unified approach. R-CNN was slow due to processing each region of interest separately. SPP-Net addressed this with spatial pyramid pooling to process regions together. Fast R-CNN further improved speed by jointly training the detector. A unified approach aims to detect objects without extracting regions of interest by dividing the image into grids and predicting objects within each grid.
1) The document discusses using high-performance distributed computing to solve large-scale multimedia problems like analyzing vast amounts of video footage.
2) It describes Parallel-Horus, a tool that makes parallel and distributed computing transparent to users through sequential programming patterns and lazy parallelization.
3) Experiments show Parallel-Horus can achieve real-time performance for tasks like color-based object recognition by distributing the work across multiple computer clusters.
Neutron noise-based core monitoring techniques are being developed to detect anomalies in operating nuclear reactors. The CORTEX project aims to advance these techniques by developing simulation tools, validating them with experiments, and applying advanced signal processing methods. This will help characterize anomalies and their root causes to improve reactor monitoring and safety. The CORTEX consortium includes experts from 17 European organizations and will work towards demonstrating these techniques using data from operating plants.
This document proposes algorithms for floorplanning reconfigurable functional units on reconfigurable architectures. It aims to minimize fragmentation while considering device heterogeneity, reconfiguration capabilities, and inter-unit communication. The approach partitions a task graph into reconfigurable regions, floorplans units within each region, and then floorplans the regions on the device using simulated annealing. Experimental results show improvements over existing approaches in area usage and communication performance.
With the HPC Cloud facility, SURFsara offers self-service, dynamically scalable and fully configurable HPC systems to the Dutch academic community. Users have, for example, a free choice of operating system and software.
The HPC Cloud offers full control over a HPC cluster, with fast CPUs and high memory nodes and it is possible to attach terabytes of local storage to a compute node. Because of this flexibility, users can fully tailor the system for a particular application. Long-running and small compute jobs are equally welcome. Additionally, the system facilitates collaboration: users can share control over their virtual private HPC cluster with other users and share processing time, data and results. A portal with wiki, fora, repositories, issue system, etc. is offered for collaboration projects as well.
Real-Time Analysis of Streaming Synchotron Data: SCinet SC19 Technology Chall...Globus
This document describes a real-time workflow for analyzing streaming synchrotron data using high-performance computing resources. Synchrotron experiments produce large amounts of data that need to be reconstructed in real-time. The workflow includes data acquisition from multiple simulated beamlines, distribution to compute nodes, tomographic reconstruction using TraceX, denoising using TomoGAN, and visualization of results. A demonstration of this workflow is being run on Argonne Leadership Computing Facility's Theta supercomputer to process streaming data from 16,000 cores in real-time and provide reconstructed volumes and feedback to experiments.
The document discusses the National Research Platform (NRP), specifically the 4th NRP workshop. It provides an overview of NRP's Nautilus, a multi-institution hypercluster connected by optical networks across 25 partner campuses. In 2022, Nautilus comprised ~200 computing nodes and 4000TB of rotating storage. The document highlights several large research projects from different domains that utilize Nautilus, including particle physics, telescopes, biomedical applications, earth sciences, and visualization. These applications demonstrate how Nautilus enables data-intensive and collaborative multi-campus research at national scale.
How to build a TraP: An image-plane transient-discovery toolTim Staley
There are three main points summarized:
1. There are many interesting slow radio transients that could be detected through imaging surveys like accretion flares, orphan gamma-ray bursts, and flare stars.
2. Radio surveys are increasing in sensitivity and field of view by orders of magnitude with instruments like LOFAR, enabling the detection of more rare transient events.
3. TraP is a transient detection pipeline that works by extracting sources from radio images, matching to known sources, identifying new bright sources, analyzing light curves, and making the results accessible through a user-friendly web interface.
This document summarizes a research paper that developed a GPU-based algorithm for CT image reconstruction from undersampled and noisy projection data. The algorithm uses an algebraic reconstruction method and implements the LSQR iterative solver on a GPU using CUDA programming. By taking advantage of massive parallel processing on the GPU, the algorithm is able to reconstruct CT images with higher resolution than previous methods without losing image quality or increasing computation time significantly. The paper presents the mathematical model, reconstruction algorithm steps, and GPU implementation details.
Evaluating Classification Algorithms Applied To Data Streams Esteban DonatoEsteban Donato
This document summarizes and evaluates several algorithms for classification of data streams: VFDTc, UFFT, and CVFDT. It describes their approaches for handling concept drift, detecting outliers and noise. The algorithms were tested on synthetic data streams generated with configurable attributes like drift frequency and noise percentage. Results show VFDTc and UFFT performed best in accuracy, while CVFDT and UFFT were fastest. The study aims to help choose algorithms suitable for different data stream characteristics like gradual vs sudden drift or frequent vs infrequent drift.
Toward Real-Time Analysis of Large Data Volumes for Diffraction Studies by Ma...EarthCube
Talk at the EarthCube End-User Domain Workshop for Rock Deformation and Mineral Physics Research.
By Martin Kunz, Lawrence Berkeley National Laboratory
Real-Time Pedestrian Detection Using Apache Storm in a Distributed Environment csandit
This document discusses real-time pedestrian detection using Apache Storm in a distributed environment. It proposes a methodology that distributes video frames across multiple nodes ("bolts") for parallel processing. Each bolt detects pedestrians in a different region of the frame. Only detection results, not full frames, are transmitted to reduce overhead. The methodology is implemented using Apache Storm and evaluated on a test system with improved processing speed. Pedestrian detection has applications in safety and analytics.
REAL-TIME PEDESTRIAN DETECTION USING APACHE STORM IN A DISTRIBUTED ENVIRONMENTcscpconf
In general, a distributed processing is not suitable for dealing with image data stream due to the
network load problem caused by communications of frames. For this reason, image data stream
processing has operated in just one node commonly. However, we need to process image data
stream in a distributed environment in a big data era due to increase in quantity and quality of
multimedia data. In this paper, we shall present a real-time pedestrian detection methodology in
a distributed environment which processes image data stream in real-time on Apache Storm
framework. It achieves sharp speed up by distributing frames onto several nodes called bolts,
each of which processes different regions of image frames. Moreover, it can reduce the
overhead caused by synchronization by computation bolts which returns only the processing
results to the merging bolts.
This document provides information on several remote sensing projects from IEEE 2015. It lists the titles, languages, and abstracts for 8 projects related to classification and analysis of hyperspectral and multispectral images. The projects focus on techniques such as sparse representation in tangent space, Gabor feature-based collaborative representation, level set evolutions for object extraction, and dimension reduction using spatial and spectral regularization.
This document describes algorithms for detecting single radio pulses in real-time using graphics processing units (GPUs). It presents two new algorithms that use incomplete sets of boxcar filters to detect pulses at accelerated speeds with minimal signal loss. The algorithms were tested on simulated data and were found to process data 266-500 times faster than real-time on GPUs, detecting pulses with a mean 7% reduction in signal power.
This document summarizes an experiment that tested optical flow-based navigation using a robot equipped with a webcam. Motion detection filters like correlation and Gabor filters were developed and applied to image sequences to detect optical flow. The filters were implemented in optical flow navigation programs for an iRobot robot. The robot was able to navigate a textured environment but struggled with obstacles and corners. Future work could include faster computation using GPUs or wider field of view cameras to improve navigation abilities.
Image Fusion Ehancement using DT-CWT TechniqueIRJET Journal
This document summarizes research on using the dual tree complex wavelet transform (DT-CWT) technique for image fusion. It begins with an abstract describing image fusion algorithms and comparing DT-CWT, discrete wavelet transform (DWT), and a basic fusion algorithm. It then provides background on image fusion, wavelet transforms, the proposed DT-CWT method, and performance metrics like peak signal-to-noise ratio and mean squared error. Simulation results show that DT-CWT yields higher PSNR and lower MSE than DWT and the basic algorithm, indicating better fusion quality.
DuraMat CO1 Central Data Resource: How it started, how it’s going …Anubhav Jain
The document summarizes several projects developed as part of the DuraMat CO1 Central Data Resource initiative to analyze photovoltaic performance and degradation data. A secure data portal was developed that currently hosts data from 239 users and 271 datasets. Software tools were also created, such as pvAnalytics for data cleaning and filtering, pvOps for operational and maintenance data analysis, and pv-vision for electroluminescence image analysis. These open source tools are publicly available and have helped advance the analysis of PV degradation through access to larger datasets. Overall, the projects have established a foundation for ongoing collaborative research on PV performance and lifetime under DuraMat 2.0.
Accelerating the Experimental Feedback Loop: Data Streams and the Advanced Ph...Ian Foster
The Advanced Photon Source (APS) at Argonne National Laboratory produces intense beams of x-rays for scientific research. Experimental data from the APS is growing dramatically due to improved detectors and a planned upgrade. This is creating data and computation challenges across the entire experimental process. Efforts are underway to accelerate the experimental feedback loop through automated data analysis, optimized data streaming, and computer-steered experiments to minimize data collection. The goal is to enable real-time insights and knowledge-driven experiments.
IJCER (www.ijceronline.com) International Journal of computational Engineerin...ijceronline
1) The document proposes a mathematical model and optimization service to predict the optimal number of parallel TCP streams needed to maximize data throughput in a distributed computing environment.
2) It develops a novel model that can predict the optimal number using only three data points, and implements this service in the Stork Data Scheduler.
3) Experimental results show the optimized transfer time using this prediction and optimization service is much less than without optimization in most cases.
This document discusses Motaz El Saban's research experience and interests which focus on analyzing, modeling, learning from, and predicting digital media content such as text, images, and speech. Some key areas of research include real-time video stitching, annotating mobile videos, object and activity recognition from videos, and facial expression recognition using deep learning techniques. The document also outlines El Saban's educational background and provides an agenda for his upcoming presentation.
This document describes Emanuele Panigati's doctoral dissertation on the SuNDroPS system for managing semantic and dynamic data in pervasive systems. It provides an overview of SuNDroPS and its components for processing streaming and historical data, including Context-ADDICT for querying heterogeneous data sources and PerLa and Tesla for information flow processing. It also describes how SuNDroPS was tested in the motivating Green Move vehicle sharing scenario.
3D-PTV is a 3D Particle Tracking Velocimetry experimental technique used in the experimental research of turbulence. main source of information is http://ptvwiki.netcipia.net
Ähnlich wie Real time extension for 3D-PTV Lagrangian measurements of turbulent canopy flow in a wind tunnel (20)
Embedded machine learning-based road conditions and driving behavior monitoringIJECEIAES
Car accident rates have increased in recent years, resulting in losses in human lives, properties, and other financial costs. An embedded machine learning-based system is developed to address this critical issue. The system can monitor road conditions, detect driving patterns, and identify aggressive driving behaviors. The system is based on neural networks trained on a comprehensive dataset of driving events, driving styles, and road conditions. The system effectively detects potential risks and helps mitigate the frequency and impact of accidents. The primary goal is to ensure the safety of drivers and vehicles. Collecting data involved gathering information on three key road events: normal street and normal drive, speed bumps, circular yellow speed bumps, and three aggressive driving actions: sudden start, sudden stop, and sudden entry. The gathered data is processed and analyzed using a machine learning system designed for limited power and memory devices. The developed system resulted in 91.9% accuracy, 93.6% precision, and 92% recall. The achieved inference time on an Arduino Nano 33 BLE Sense with a 32-bit CPU running at 64 MHz is 34 ms and requires 2.6 kB peak RAM and 139.9 kB program flash memory, making it suitable for resource-constrained embedded systems.
An improved modulation technique suitable for a three level flying capacitor ...IJECEIAES
This research paper introduces an innovative modulation technique for controlling a 3-level flying capacitor multilevel inverter (FCMLI), aiming to streamline the modulation process in contrast to conventional methods. The proposed
simplified modulation technique paves the way for more straightforward and
efficient control of multilevel inverters, enabling their widespread adoption and
integration into modern power electronic systems. Through the amalgamation of
sinusoidal pulse width modulation (SPWM) with a high-frequency square wave
pulse, this controlling technique attains energy equilibrium across the coupling
capacitor. The modulation scheme incorporates a simplified switching pattern
and a decreased count of voltage references, thereby simplifying the control
algorithm.
Null Bangalore | Pentesters Approach to AWS IAMDivyanshu
#Abstract:
- Learn more about the real-world methods for auditing AWS IAM (Identity and Access Management) as a pentester. So let us proceed with a brief discussion of IAM as well as some typical misconfigurations and their potential exploits in order to reinforce the understanding of IAM security best practices.
- Gain actionable insights into AWS IAM policies and roles, using hands on approach.
#Prerequisites:
- Basic understanding of AWS services and architecture
- Familiarity with cloud security concepts
- Experience using the AWS Management Console or AWS CLI.
- For hands on lab create account on [killercoda.com](https://killercoda.com/cloudsecurity-scenario/)
# Scenario Covered:
- Basics of IAM in AWS
- Implementing IAM Policies with Least Privilege to Manage S3 Bucket
- Objective: Create an S3 bucket with least privilege IAM policy and validate access.
- Steps:
- Create S3 bucket.
- Attach least privilege policy to IAM user.
- Validate access.
- Exploiting IAM PassRole Misconfiguration
-Allows a user to pass a specific IAM role to an AWS service (ec2), typically used for service access delegation. Then exploit PassRole Misconfiguration granting unauthorized access to sensitive resources.
- Objective: Demonstrate how a PassRole misconfiguration can grant unauthorized access.
- Steps:
- Allow user to pass IAM role to EC2.
- Exploit misconfiguration for unauthorized access.
- Access sensitive resources.
- Exploiting IAM AssumeRole Misconfiguration with Overly Permissive Role
- An overly permissive IAM role configuration can lead to privilege escalation by creating a role with administrative privileges and allow a user to assume this role.
- Objective: Show how overly permissive IAM roles can lead to privilege escalation.
- Steps:
- Create role with administrative privileges.
- Allow user to assume the role.
- Perform administrative actions.
- Differentiation between PassRole vs AssumeRole
Try at [killercoda.com](https://killercoda.com/cloudsecurity-scenario/)
artificial intelligence and data science contents.pptxGauravCar
What is artificial intelligence? Artificial intelligence is the ability of a computer or computer-controlled robot to perform tasks that are commonly associated with the intellectual processes characteristic of humans, such as the ability to reason.
› ...
Artificial intelligence (AI) | Definitio
Electric vehicle and photovoltaic advanced roles in enhancing the financial p...IJECEIAES
Climate change's impact on the planet forced the United Nations and governments to promote green energies and electric transportation. The deployments of photovoltaic (PV) and electric vehicle (EV) systems gained stronger momentum due to their numerous advantages over fossil fuel types. The advantages go beyond sustainability to reach financial support and stability. The work in this paper introduces the hybrid system between PV and EV to support industrial and commercial plants. This paper covers the theoretical framework of the proposed hybrid system including the required equation to complete the cost analysis when PV and EV are present. In addition, the proposed design diagram which sets the priorities and requirements of the system is presented. The proposed approach allows setup to advance their power stability, especially during power outages. The presented information supports researchers and plant owners to complete the necessary analysis while promoting the deployment of clean energy. The result of a case study that represents a dairy milk farmer supports the theoretical works and highlights its advanced benefits to existing plants. The short return on investment of the proposed approach supports the paper's novelty approach for the sustainable electrical system. In addition, the proposed system allows for an isolated power setup without the need for a transmission line which enhances the safety of the electrical network
Applications of artificial Intelligence in Mechanical Engineering.pdfAtif Razi
Historically, mechanical engineering has relied heavily on human expertise and empirical methods to solve complex problems. With the introduction of computer-aided design (CAD) and finite element analysis (FEA), the field took its first steps towards digitization. These tools allowed engineers to simulate and analyze mechanical systems with greater accuracy and efficiency. However, the sheer volume of data generated by modern engineering systems and the increasing complexity of these systems have necessitated more advanced analytical tools, paving the way for AI.
AI offers the capability to process vast amounts of data, identify patterns, and make predictions with a level of speed and accuracy unattainable by traditional methods. This has profound implications for mechanical engineering, enabling more efficient design processes, predictive maintenance strategies, and optimized manufacturing operations. AI-driven tools can learn from historical data, adapt to new information, and continuously improve their performance, making them invaluable in tackling the multifaceted challenges of modern mechanical engineering.
Batteries -Introduction – Types of Batteries – discharging and charging of battery - characteristics of battery –battery rating- various tests on battery- – Primary battery: silver button cell- Secondary battery :Ni-Cd battery-modern battery: lithium ion battery-maintenance of batteries-choices of batteries for electric vehicle applications.
Fuel Cells: Introduction- importance and classification of fuel cells - description, principle, components, applications of fuel cells: H2-O2 fuel cell, alkaline fuel cell, molten carbonate fuel cell and direct methanol fuel cells.
2. Outline
• What is the three-dimensional particle tracking velocimetry
• What is the main contribution to turbulence research
• What are the limitations and possible solutions
• The (only) working solution example - Ron Shnapp
3. The take-home message
• You can do it and you better do it
• Real-time image analysis on a dedicated hardware is a key for
long-recording, high-speed 3D-PTV
• Paradigms about 3D-PTV changes from in-lab table-top
experiments to wind tunnel and field experiments
• You can readily get data sets of millions of tracer trajectories
like the one recorded in a canopy flow model in an
environmental wind tunnel, providing Lagrangian velocity and
acceleration distributions in an urban canopy flow model
Details are in the pre-print “Real-time extension ...” by
Shnapp et al. arXiv:1806.04975
4. Let’s follow the Lagrangian path
“Marianthe” invited people inside turbulent forms to experi-
ence them as if they were a particle borne along in the flow.
Athena Tacha (1985), “Nautilus” by Philip Ball
17. Works very well for lab experiments
Raw image - binarized image - blobs marked on the original
image.
18. Real life is not like this
Raw 3D-PTV image the wind tunnel experiments: Back-
ground -¿ binary image after background subtraction, and
detection using a local adaptive filter
22. And now we are ready for the Environmental Wind Tunnel
23. Open source software suite, all on Github
• library, ‘liboptv‘, ANSI C
• PyPTV GUI for liboptv in Python
• FlowTracks - trajectories database management (see Meller
and Liberzon 2016)
• BlobRecorder - proprietary hardware/customized software (see
Shnapp et al. arxiv)