One of the most powerful features of PostgreSQL is its diversity of procedural languages, but with that diversity comes a lot of options.
Did you ever wonder:
- What all of those options are on the CREATE FUNCTION statement?
- How do they affect my application?
- Does my choice of procedural language affect the performance of my statements?
- Should I create a single trigger with IF statements or several simple triggers?
- How do I debug my code?
- Can I tell which line in my function is taking all of the time?
5. functions=# SELECT airport FROM bird_strikes LIMIT 5;
airport
--------------------------
NEWARK LIBERTY INTL ARPT
UNKNOWN
DENVER INTL AIRPORT
CHICAGO O'HARE INTL ARPT
JOHN F KENNEDY INTL
(5 rows)
Source: http://wildlife.faa.gov/
Sample Data
6. functions=# SELECT count(*)
functions-# FROM bird_strikes
functions-# WHERE get_iata_code_from_abbr_name(airport) =
'LAX';
count
-------
850
(1 row)
Time: 13490.611 ms
Data Formatting Functions
7. functions=# EXPLAIN ANALYZE SELECT count(*) FROM bird_strikes ...
QUERY PLAN
------------------------------------------------------------------------
Aggregate (cost=29418.79..29418.80 rows=1 width=0) (actual
time=13463.628..13463.629 rows=1 loops=1)
-> Seq Scan on bird_strikes (cost=0.00..29417.55 rows=497 width=0)
(actual time=15.721..13463.293 rows=850 loops=1)
Filter: ((get_iata_code_from_abbr_name(airport))::text =
'LAX'::text)
Rows Removed by Filter: 98554
Planning time: 0.124 ms
Execution time: 13463.682 ms
(6 rows)
Check Performance
8. functions=# set track_functions = 'pl';
SET
functions=# select * from pg_stat_user_functions;
(No rows)
functions=# SELECT count(*) FROM bird_strikes ...
-[ RECORD 1 ]
count | 850
Track Function Usage
11. CREATE OR REPLACE FUNCTION get_iata_code_from_name(airport_name varchar)
RETURNS varchar AS
$$
DECLARE
working_name varchar;
code varchar := null;
BEGIN
working_name := upper(airport_name);
EXECUTE $__$ SELECT iata_code
FROM airports
WHERE upper(name) LIKE $1
$__$
INTO code
USING working_name;
RETURN code;
END;
$$ LANGUAGE plpgsql;
14. ● Be careful when you have a
function call another function
– May lead to difficult to diagnose
performance problems
● Be careful when a function is used
in a WHERE clause
– For sequential scans, it may
execute once per row in the table
16. CREATE TYPE airport_regions AS (airport_name varchar,
airport_continent varchar,
airport_country varchar,
airport_state varchar);
CREATE OR REPLACE FUNCTION get_airport_regions()
RETURNS SETOF airport_regions AS
$$
BEGIN
RETURN QUERY SELECT name::varchar, continent::varchar,
iso_country::varchar,
split_part(iso_region, '-', 2)::varchar
FROM airports;
END;
$$ LANGUAGE plpgsql;
Set Returning Functions
17. functions=# SELECT b.num_wildlife_struck
FROM bird_strikes b, state_code s,
get_airport_regions() r
WHERE b.origin_state = s.name
AND s.abbreviation = r.airport_state
AND r.airport_continent = 'NA';
num_wildlife_struck
---------------------
…
Time: 48507.635 ms
18. QUERY PLAN
-----------------------------------------------------------------------------------------------------------
Nested Loop (cost=42.10..318.77 rows=1972 width=2) (actual time=43.468..38467.229 rows=60334427 loops=1)
-> Hash Join (cost=12.81..14.51 rows=1 width=9) (actual time=43.284..58.007 rows=21488 loops=1)
Hash Cond: ((s.abbreviation)::text = (r.airport_state)::text)
-> Seq Scan on state_code s (cost=0.00..1.50 rows=50 width=12) (actual time=0.007..0.045 rows=50
loops=1)
-> Hash (cost=12.75..12.75 rows=5 width=32) (actual time=43.264..43.264 rows=25056 loops=1)
Buckets: 1024 Batches: 1 Memory Usage: 857kB
-> Function Scan on get_airport_regions r (cost=0.25..12.75 rows=5 width=32) (actual
time=34.050..39.650 rows=25056 loops=1)
Filter: ((airport_continent)::text = 'NA'::text)
Rows Removed by Filter: 21150
-> Bitmap Heap Scan on bird_strikes b (cost=29.29..288.48 rows=1578 width=10) (actual
time=0.445..1.343 rows=2808 loops=21488)
Recheck Cond: ((origin_state)::text = (s.name)::text)
Heap Blocks: exact=31639334
-> Bitmap Index Scan on bird_strikes_state (cost=0.00..28.89 rows=1578 width=0) (actual
time=0.285..0.285 rows=2808 loops=21488)
Index Cond: ((origin_state)::text = (s.name)::text)
Planning time: 0.742 ms
Execution time: 40447.925 ms
(16 rows)
Time: 40449.209 ms
19. CREATE OR REPLACE FUNCTION get_airport_regions()
RETURNS SETOF airport_regions AS
$$
BEGIN
RETURN QUERY SELECT name::varchar, continent::varchar,
iso_country::varchar,
split_part(iso_region, '-', 2)::varchar
FROM airports;
END;
$$ LANGUAGE plpgsql
ROWS 46206
COST 600000;
21. ● When using set returning functions as
tables, the row and cost estimates are
usually way off
– Default ROWS: 1000
– Default COST: 100
● Note: COST is in units of
cpu_operator_cost which is 0.0025
22. ● Do not use functions to mask a
bad data model
● Use functions to help load the data
into the correct format
23. Table Partitioning
● Usually done for performance
● Uses check constraints and inherited tables
● Triggers are preferred over rules so COPY can be used
● Trigger functions used to move the data to the correct
child table
25. CREATE OR REPLACE FUNCTION partition_trigger() RETURNS trigger AS $$
DECLARE
partition int;
BEGIN
partition = NEW.key % 5;
EXECUTE 'INSERT INTO trigger_test_' || partition || ' VALUES
(($1).*)' USING NEW;
RETURN NULL;
END;
$$ LANGUAGE plpgsql;
CREATE TRIGGER partition_trigger BEFORE INSERT ON trigger_test
FOR EACH ROW EXECUTE PROCEDURE partition_trigger();
Dynamic Trigger
26. CREATE OR REPLACE FUNCTION partition_trigger() RETURNS trigger AS $$
BEGIN
CASE NEW.key % 5
WHEN 0 THEN
INSERT INTO trigger_test_0 VALUES (NEW.*);
WHEN 1 THEN
INSERT INTO trigger_test_1 VALUES (NEW.*);
WHEN 2 THEN
INSERT INTO trigger_test_2 VALUES (NEW.*);
WHEN 3 THEN
INSERT INTO trigger_test_3 VALUES (NEW.*);
WHEN 4 THEN
INSERT INTO trigger_test_4 VALUES (NEW.*);
END CASE;
RETURN NULL;
END;
$$ LANGUAGE plpgsql;
Case Statement
27. ● 16% performance gain using CASE Statement
● Tested inserting 100,000 rows
Dynamic Trigger Case Trigger
3200
3400
3600
3800
4000
4200
4400
Performance of Partition Triggers
28. Trigger Overhead
● Triggers get executed when an event
happens in the database
– INSERT, UPDATE, DELETE
● Event Triggers fire on DDL
– CREATE, DROP, ALTER
29. CREATE UNLOGGED TABLE trigger_test (
key serial primary key,
value varchar,
insert_ts timestamp,
update_ts timestamp
);
INSERTS.pgbench
INSERT INTO trigger_test (value) VALUES (‘hello’);
31. CREATE FUNCTION empty_trigger() RETURNS trigger AS $$
BEGIN
RETURN NEW;
END;
$$ LANGUAGE plpgsql;
CREATE TRIGGER empty_trigger BEFORE INSERT OR UPDATE ON
trigger_test
FOR EACH ROW EXECUTE PROCEDURE empty_trigger();
44. ● Think things through before
adding server side code
● Performance test your functions
● Don't use a procedural language
just because it's cool
– Use the right tool for the job