5. Will Aoe_Import be the fastest
product importer around?
YES, of course!
Well, maybe…
Actually, Aoe_Import is only a
XML Importer “Framework”. It’s
up to you to decide how to handle
the xml snippets…
6. for large XML files
XML! Not CSV.
full flexibility in
processor implementation
Aoe_Import
multi-thread support!
Subscribe your
“Processors” to xpaths
Stream processing
(XMLReader)
“event” driven
9. Beat the memory Leak
by forking
Waiting for other
thread to terminate
Threading
overhead
Process
import
10. Forking?
In PHP?
$pid = pcntl_fork();
if ($pid) {
// parent process runs what is here
echo "parentn";
} else {
// child process runs what is here
echo "childn";
}
14. No imports are processed in the main thread.
So there’s no memory leak happing here
Main thread
memory
Memory limit
time
Create process
collection
Waiting for other thread to terminate
Threading Process imports in process collection
overhead
Forks
Every fork starts with the low
memory footprint of the main thread
Find the number of imports
that can be processed at a
time without hitting the memory limit