An introduction to Augustus, an open source scoring engine for statistical and data mining models based on the Predictive Model Markup Language (PMML). Augustus is able to produce and consume models with 10,000s of segments. Developed by Open Data Group, written in Python, PMML 4.0 compliant and freely available.

1. Introduction to Augustus OVERVIEW Open Data Group September 17, 2009

2. Website and Community Augustus is an open source scoring engine for statistical and data mining models based on the Predictive Model Markup Language (PMML). It is written in Python and is freely available. http://augustus.googlecode.com

15. Development and Use Cycle 2. Model schema 1. Data Inputs

16. Running Augustus 3. Obtain new model with Producer 4. Score with Consumer

22. Result of Scoring

24. Work Flow Overview

25. Auto: Weighted Batch Using the Baseline for Training: $ cd WeightedBatch `-- scripts |-- consume.py |-- postprocess.py `-- produce.py http://code.google.com/p/augustus/source/browse/#svn/trunk/examples/auto/WeightedBatch

26. Input for the Producer The Producer takes the training data set. In the code, we have declared how we want to test the data import augustus.modellib.baseline.producer.Producer as Producer def makeConfigs(inFile, outFile, inPMML, outPMML): #open data file inf = uni.UniTable().fromfile(inFile) #start the configuration file test = ET.SubElement(root, "test") test.set("field", "Automaker") test.set("weightField", "Count") test.set("testStatistic", "dDist") test.set("testType", "threshold") test.set("threshold", "0.475")

27. Input for the Producer Continued # use a discrete distribution model for test baseline = ET.SubElement(test, "baseline") baseline.set("dist", "discrete") baseline.set("file", str(inFile)) baseline.set("type", "UniTable") # create the segmentation declarations for the two fields at this level ''' Taken out for the example, other Use Cases will focus on Segments segmentation = ET.SubElement(test, "segmentation") makeSegment(inf, segmentation, "Color") ''' #output the configuration file tree = ET.ElementTree(root) tree.write(outFile)

28. Running the Producer( Training) $ cd scripts $ python2.5 produce.py -f wtraining.nab -t20 (0.000 secs) Beginning timing (0.000 secs) Creating configuration file (0.001 secs) Creating input PMML file (0.001 secs) Starting producer (0.000 secs) Inputting configurations (0.001 secs) Inputting model (0.008 secs) Collecting stats for baseline distribution (0.011 secs) Events 20.067% processed (0.009 secs) Events 40.134% processed (0.009 secs) Events 60.201% processed (0.009 secs) Events 80.268% processed (0.009 secs) Events 100.000% processed (0.000 secs) Making test distributions from statistics (0.002 secs) Outputting PMML (0.062 secs) Lifetime of timer

29. Model generated by the Producer <PMML version=&quot;3.1&quot;> <Header copyright=&quot; &quot; /> < DataDictionary > < DataField dataType=&quot;string&quot; name=&quot;Automaker&quot; optype=&quot;categorical&quot; /> < DataField dataType=&quot;string&quot; name=&quot;Color&quot; optype=&quot;categorical&quot; /> < DataField dataType=&quot;float&quot; name=&quot;Count&quot; optype=&quot;continuous&quot; /> </ DataDictionary > < BaselineModel functionName=&quot;baseline&quot;> < MiningSchema > < MiningField name=&quot;Automaker&quot; /> < MiningField name=&quot;Color&quot; /> < MiningField name=&quot;Count&quot; /> </ MiningSchema > </ BaselineModel > </PMML>

31. Producer Output The training step used the code in producer.py to generate a model and get expected results. Training generated the following files: . |-- consumer | `-- wtraining.nab.pmml MODEL WITH EXPECTED VALUES BASED ON THE TRAINING DATA `-- producer |-- wtraining.nab.pmml BASELINE DATA, DATA DICTIONARY, MINING SCHEMA `-- wtraining.nab.xml MODEL FILE USED FOR TRAINING

34. Running the Consumer cd script $ python2.5 consume.py -b wtraining.nab -f wscoring.nab Ready to score . |-- consumer | |-- wscoring.nab.wtraining.nab.xml | `-- wtraining.nab.pmml |-- postprocess | `-- wscoring.nab.wtraining.nab.xml `-- producer |-- wtraining.nab.pmml `-- wtraining.nab.xml This examples generates a report in the post process directory.

35. Consumer (Scoring) output $ cat consumer/wscoring.nab.wtraining.nab.xml <pmmlDeployment> <inputData> <readOnce /> <batchScoring /> <fromFile name=&quot;../data/wscoring.nab&quot; type=&quot;UniTable&quot; /> </inputData> <inputModel> <fromFile name=&quot;../consumer/wtraining.nab.pmml&quot; /> </inputModel> <output> <report name=&quot;report&quot;> <toFile name=&quot;../postprocess/wscoring.nab.wtraining.nab.xml&quot; /> <outputRow name=&quot;event&quot;> <score name=&quot;score&quot; /> <alert name=&quot;alert&quot; /> <segments name=&quot;segments&quot; /> </outputRow> </report> </output> </pmmlDeployment>

36. Scoring Report $ cat postprocess/ wscoring.nab.wtraining.nab.xml <report> < event > < score >0.471458430077</ score > < alert >True</ alert > < Segments ></ Segments > </ event > </report>