Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Hyperbatch (LoteRapido) - Punta Dreamin' 2017
1. HyperBatch (LoteRapido)
Daniel Peter
Lead Applications Engineer, Kenandy
Salesforce MVP
Bay Area Salesforce Developer User Group Organizer
22x certified
dan@danpeter.com
@danieljpeter
A Hyper-Fast Batchable Interface for Salesforce
3. Why HyperBatch?
Speed!
Case study: Account / Contact Batches
• Prerequisite: 121K Accounts already in the system
• CreateContactsBatch: Creates 3 Contacts for each Account with a random
“probability” field for each. 363k Contacts total.
• UpdateAccountsBatch: For each Account, update the highest and lowest
probability on the Account by querying the child Contacts. Get the overall
highest and lowest probability across all the Accounts.
• DeleteContactsBatch: Delete all of the Contacts in the system. Keep a running
total of how many get deleted.
15. Why HyperBatch?
Concurrency
Row lock behavior
• Apex Batch: default is a failed batch execution. Retry logic can be built, but it
will likely exceed the transaction limits.
• HyperBatch: row locks retry automatically until the transaction succeeds. Each
re-attempt gets a new context!
16. How it Works
Summary
• HyperBatch interface that mimics the Database.Batchable interface.
• Browser orchestration for selecting jobs and running them on-demand.
• Lightning Design System, Visualforce (Lightning Components would be a data bottleneck).
• AJAX toolkit for PK chunking the query locator.
• Wrapping requests in unique identifiers for closed loop execution – JavaScript function binding.
• Each execute can return some state of type Object, it can be anything.
• Async
• Parallel remote actions fire the qeueables for the batch executions methods. (Not serial!)
• JavaScript polls for the status of the qeueables, waiting for them to complete.
• Execute state is stored in a custom object, and a list of them is returned to the finish() method,
then they are deleted.
• Sync
• Parallel remote actions fire the synchronous transactions. (Not serial!)
• Execute state is returned synchronously and stored in JavaScript in the browser until they are
all complete.
24. Roadmap
• Throttle requests (x number of open connections)
• Enhance the user interface
• Support custom iterators instead of just query locator
• Support simple data operations like update a field or delete records
without having to write Apex
• Chunk in 2 dimensions: (Parent Id, then Record Id) to avoid row
lock errors
25. Resources
• Salesforce Developer Blogs
• “Data Chunking Techniques for Massive Orgs“ by Daniel Peter
(https://developer.salesforce.com/blogs/developer-relations/2015/11/pk-chunking-
techniques-massive-orgs.html)
• “French Touch Dreamin’ – The first community-led event in France” by Philippe Ozil
(https://developer.salesforce.com/blogs/developer-relations/2016/11/french-touch-
dreamin-first-community-led-event-france.html)
• Presentation from Forcelandia 2016: “PK Chunking – Divide and conquer massive objects
in Salesforce” (http://www.slideshare.net/danieljpeter/forcelandia-2016-pk-chunking)
• GitHub repo: HyperBatch (https://github.com/danieljpeter/HyperBatch)