2. PROBLEM STATEMENT
WHY TESTING IS HARD
▸ Need to test on live traffic
▸ Testing environment might be less powerful (e.g. a VM)
▸ Experimental machines can fail or die
▸ Need to compare all results
8. ARCHITECTURE
MAIN FEATURES
▸ Reply immediately when get results from the main cluster
▸ Fast and low latency architecture
▸ Can use multiple compare result scripts
▸ Compare scripts could use all API functions from rspamd
10. ARCHITECTURE
ENCRYPTION PROXY
▸ Encrypt with HTTPCrypt:
▸ low latency (0 RTT before data sending)
▸ zero copy
▸ provable secure
▸ simple keys management
▸ Can open local files and send encrypted data stream
▸ Each cluster can have its own unique encryption key
▸ Local keys are rotated frequently
12. ARCHITECTURE
LOAD BALANCING
▸ Send certain amount of traffic to each testing cluster
▸ Balance within each cluster:
▸ balancing schemes: round-robin, master-slave, random
▸ each server can have its own priority
▸ can detect if an upstream is down
▸ lazily resolve upstream names (DNS balancing)
14. ARCHITECTURE
FOREIGN EXTERNAL SCANNERS
▸ Can scan external scanners, e.g. SA or Cloudmark
▸ Can evaluate their efficiency comparing to rspamd
▸ Use Lua filter to parse external scanners results
15. COMPARE EXAMPLES
AN EXAMPLE OF COMPARISON SCRIPT
return function(results)
local log = require "rspamd_logger"
for k,v in pairs(results) do
if type(v) == 'table' then
log.infox("%s: %s", k, v['default']['score'])
else
log.infox("err: %s: %s", k, v)
end
end
end
16. FUTURE PLANS
POTENTIAL FEATURES
▸ Balance not merely HTTP but also SMTP
▸ Perform retries when master connection fails somehow
▸ Use mirrors results if the whole stable cluster is dead
▸ Location based balancing (select the nearest or the fastest
server among possible choices)