1. Building Large Scale Digital Asset
Repositories with AEM DAM & Scene7
Devang Shah, I-Cubed

2. INTRODUCTION
About Me:
• Platform Architect, Emerging Technologies at I-Cubed
• Prior Life:
• Acquity Group, Part of Accenture Interactive
• ContextMine
• eFORCE Global
• Lived in Reading & Swindon during early stages of my career
“I-Cubed helps enterprises connect with their customers across multiple online
channels through the implementation, support and training of Digital Experience
Management solutions”

3. The Growing Demand for Rich Media
Web Site Then
& Now
With the growing demand of
digital content and rich media;
Digital Asset Management (DAM)
systems allow for organizing,
enhancing, sharing and distributing
of digital assets

4. Digital Asset Management - Content Life Cycle & Use Cases
DAM: Asset Storage
Asset Creation:
• Support for wide
variety of formats
• Metadata Embed
Support
• Encoding / Decoding
Asset Management:
• Advanced Metadata & Tagging
• Search & Browse
• Lock, Publish, Version, Archive
• Workflow support
• Authentication, Authorization, DRM
Publishing & Delivery:
• Web &/or Print publishing
• Secure v/s non-secure
access
• End users: B2B or B2C
• File Serving, Streaming
• CDN
Content Producers Content Editors & Collaborators Content Consumers
Overview of Digital Asset Management Ecosystem

5. AEM: Digital Asset Manager (DAM) Adobe Scene7
• Digital asset management platform for
business users
• Tightly integrated with AEM Web Content
Management tool
• Support for wide range of rich media asset
types and formats
• Metadata & Workflow driven asset handling
• Integration with Adobe Creative Cloud to
allow collaboration with asset creative team
• Hosted solution for managing and publishing
dynamic media with local caching & CDN
• Dynamic generation of asset renditions (or
variations)
• Multi-screen engagement using Adaptive content
delivery
• Management and publishing of videos to web and
mobile platforms
• Integration with SiteCatalyst for enhanced tracking &
reporting
AEM DAM & Adobe Scene7: Scaling Asset Delivery
Product Highlights

6. Basic
Integration
Improved Asset
synchronization
Dynamic Media
Capabilities
CQ 5.5
• Configure scene7 cloud service
• Manual asset upload to S7
• Auto asset upload via workflow launcher
• OOTB Scene7 Asset Viewer components
AEM 5.6.1
• Configure polling importer, with support for two way
synchronization between AEM DAM & Scene7
• S7 Video Component & Options to S7 Video Presents in
AEM
• Scene7 content browser for viewing all S7 assets in AEM
AEM 6.0
• Dynamic media capabilities with option
to access S7 functionality within AEM
DAM
Integration
Highlights
AEM DAM & Adobe Scene7: Scaling Asset Delivery
Integration Overview

7. AEM DAM & Adobe Scene7: Scaling Asset Delivery
OOTB Integration Options

8. AEM DAM & Adobe Scene7: Scaling Asset Delivery
OOTB Integration Options

9. AEM DAM & Adobe Scene7: Scaling Asset Delivery
Integration Limitations & Custom Enhancements
Few Limitations of the Integration (v 5.5 / 5.6.1) & Customizations:
- Custom Scene7 Image Component & corresponding Image Tag
- Asset deletion in AEM DAM not reflected in Scene7
- Inconsistent Scene7 publish complete status update (daycare hotfix available)
- DAM Admin Interface customized to display Scene7 Upload Status

10. But, what happens when you try to
open up the flood gate .. err..
expand the DAM repository usage
without optimizing & scaling it
first?
How to Scale Asset Ingestion?

11. Images slow to get
processed
ObservationDispatcher
More than 20000 events
in the queue
Out of Memory
Exceptions
Blocked Replication
Queue
Unstable Author
Environment
Unable to size & scope
system architecture
Unable to complete
System Processes
(TarPM & GC)
Author
Challenges
Developer
Challenges
Sys Admin
Challenges
Image not getting
uploaded to S7
Scaling DAM Asset Ingestion - Challenges

12. Performance != Scalability
• Performance: Capability of a system to provide certain response time.
• Scalability: Characteristic of a system to increase performance by adding additional
resources.
• Scalability can help increase performance; but scaling system without fine tuning and
optimizing it first, may not yield much benefit and will waste system resources.
Step1: Performance Optimization & Fine Tuning
• How can we fine tune and optimize the DAM asset ingestion process?
Step 2: Scalability
• How can we scale the system so that it can handle „x‟ number of assets knowing that
it takes „y‟ number of resources per asset.
Optimize & Scale (In that order)

13. Understand How Assets Are Processed & Verify Asset Processing Steps
Question To Ask:
• Is DMA used to store videos? OR
• Is XMP metadata functionality needed?
Approach:
• If no, trim down DAM Update asset processing workflow by removing unnecessary steps
DAM Performance Optimization & Fine Tuning
Know Your Use Cases
Asset Node
Created
Trigger
Observation
Event
Event Placed
in Event
Queue
Extract Asset
Metadata
Process
Workflow
Event
Create
Thumbnails
FFMpeg Video
Processing
Create Web
Rendition
InDesign
Extraction
DAM Update Asset Workflow

14. Verify Asset Size Requirement
Resources required to process an asset is directly proportional to asset resolution
Question To Ask:
• How are assets being used? Are assets used only for web display or for Web + Print?
Approach:
• If assets are used only for web display, restrict &/or reduce asset sizes and ensure that
assets uploaded to DAM are optimized per web consumption
DAM Performance Optimization & Fine Tuning
Know Your Use Cases

15. If a rendition is used frequently and consistently across site pages:
Recommended Approach
• Generate renditions at asset creation time via asset update workflow
Benefit:
• Increase in page load performance
• Consistent access and generation of renditions
• Author can override a rendition, if needed
If a rendition is used sparingly across site pages :
Recommended Approach
• Generate rendition at run time via image processing script
Benefit:
• Reduced asset processing time and author instance resource usage
• Reduced disk space utilization
DAM Performance Optimization & Fine Tuning
Understand How Asset Renditions are Used

16. Workflow & Audit Purge Service
• Periodic deletion of archived workflow events. For version 5.5, download it from Adobe KB Base
page. Version 5.6+ built-in service.
Asset Hierarchy
• Organize asset hierarchy such a way that it limits number of direct sub nodes within a particular
node (< 1000).
Event Queue Size
• Parameter: jackrabbit.maxQueuedEvents
• Default limit is 20000, can be changed by setting value via start up script
Vertical scaling of infrastructure
• but, additional RAM and CPU may not solve all the issues, as I/O & network might all become
bottleneck
DAM Performance Optimization & Fine Tuning
Various Configuration Parameters & Other Options

17. Use Case: Constant stream of new digital asset ingestion – unfiltered and without manual
intervention. For example: Large news & media sites
Approach:
Create separate job or asset processing server which ingests & processes assets and
replicates to the main author instance.
CQ / Asset
Ingestor
CQ Author
CQ Author
CQ Publish
CQ Publish
Replication
(auto asset ingestion
Replication (editorial changes)
Architecture Scaling for Asset Processing
Separate Instance for Ingesting & Processing Assets
Scene7
Batch / Auto
Asset Upload
Process
S7 Auto
Asset
Upload

18. Use Case: High volume of author ingested data or asset ingestion requiring manual
updates.
Approach: (5.6.1) Offload execution of complex and resource intensive tasks (for example
DAM Update Asset workflow) to a separate instance.
CQ Author
CQ Author
CQ Publish
CQ Publish
Replication
CQ / Asset
Processor
Asset Process -
Task offloading
Architecture Scaling for Asset Processing
Separate Instance for Executing Process Intensive Tasks
Scene7
Batch / Auto
Asset Upload
Process
S7 Auto
Asset
Upload

19. Use Case: Large scale repositories with high storage requirements.
Options If using Scene7: Publish asset to Scene7 & Only metadata to CQ Publisher
Instances.
CQ Author
CQ Author
CQ Publish
CQ Publish
Replication
• Replicate only asset
metadata. Actual
asset will be served
via S7
Shared Data Store & Repository
Architecture Scaling for Asset Processing
Scene7
S7 Asset
Upload

20. Questions? Feel free to reach out..
Devang Shah
Platform Architect, I-Cubed
devang.shah@i-cubed.com | +1.919.615.4365