There are more and more companies have big Firebird databases, from 100Gb till 1Tb. Maintenance and optimization tasks for such databases are different from small, and database administrators need take into account several important things about big Firebird databases.
Axa Assurance Maroc - Insurer Innovation Award 2024
Life with big Firebird databases
1. Life with big Firebird databases
Alexey Kovyazin
www.IBSurgeon.com
2. 2
• Tools and consulting
• Platinum Sponsor of Firebird Project
• Founded in 2002: 12 years of
Firebird and InterBase recoveries
and consulting
• Based in Moscow, Russia
• Firebase is our Brazil partner (since
2006)
3. Big databases 3
Series of performance tests
with databases 9Gb... 30Gb... 1.7Tb
http://ib-aid.com/en/articles/firebird-performance-degradation-tests-myths-and-truth/
6. 6 What we did
• SuperServer → Classic or SuperClassic
• Tuned Firebird config parameters
Optimized config are available here:
http://ib-aid.com/en/optimized-firebird-configuration/
7. 7 The difference between big and
small databases
• Small
– Optimize your SQLs
– In case of any trouble
do backup/restore
(gbak)
• Big
– Where is the problem?
– Why it happens?
– Backup/restore takes 1
days... Will it solve the
problem or it will be
waste of time?
9. 9 Backups
• Usual backup: new backup rewrites old backup
Firebird database
New backup
Old backup
10. 10 Wrong!
• If backup fails, there will be no good old
backup, no new backup, and corrupted
database
Firebird database
New backup
Old backup
11. 11 Correct schema
Firebird database
New backup
Finished backup Old backup
Database space for backups:
(Number_of_stored_backups+1)*Size
12. 12 Gbak vs Nbackup
• NBACKUP
+ fast
- does not check data
- not reliable till 2.5.2
- delta problems
• Gbak
+ checks data
- slow (and speed
degrades)
- test restore needed
13. 13 Big databases backup schema
Requirements
● delta monitoring
● health monitoring
Requirements:
● gbak backup
● test restore
Production server Backup server
Nbackup gbak
14. 14 Big databases backup schema
FBDataGuard 1 FBDataGuard 2
Production server Backup server
Nbackup gbak
15. 15 FBDataGuard
• Backups (in correct way)
• Health checks of the database
• Performance parameters monitoring
– Transactions
– Users, Indices, database statistics
• Corruption monitoring
• Email alerts
16. 16 Health monitoring
• Critical metadata check and backup
– Can be used for recovery with FirstAID
• Database limitations checks
– Transactions limits, format limits, space limits
• Errors
– Firebird.log
17. 17 Indices maintenance
• Indices health
– Idea of statistics
– Recalculation of indices statistics
• Manual
• Automatic
20. 20 Tasks and Tools
• SQL queries and plans: FBScanner and
FBPerfMon
• Transactions: FBDataGuard (alerts, database
statistics) and IBTM (visualization)
• Database structure analysys: IBAnalyst
21. 21
Recommended setup for
monitoring
Firebird
Production server
Users
FBScanner
remote server
Administrator has:
1) FBScanner Analyzer
2) MON$Logger
3) TraceAPI
4) IBTM
5) IBAnalyst
MON$
TraceAPI
FBDataGuard
& Trace API
22. Examples of problem resolution 22
Long running active transaction — prevent
performance problem
1.Administrator got email from FBDataGuard about
transaction gap
2.IBTM shows long running active transaction
3.MON$ logger identified source of transaction, and
admin disconnected client
23. 23 Transactions monitoring
• Transaction markers (NEXT, OIT, OST, OAT)
dynamics is good indicator of problems
– Long running active transactions (stuck OAT)
– Forced Rollback (stuck OIT)
– Sweep and autosweep success
26. Examples of problem resolution 26
Bad performance — fix performance problem
– User reported problem with performance in application A
– Quick check with MON$logger does not show the
problem
– Admin analysed logs from FBScanner (permanent
monitoring) and found bad query, reported to developer,
with SQL plan and fresh gstat statistics from FBDataGuard
– Developer checked gstat and plan, found that new index
is needed
– Admin applied index, performance is good again
27. 27 SQL Queries and Plans
• For all Firebird versions — FBScanner
– Works as a proxy
– Connection string changes required
– Can be installed on remote server
• For Firebird 2.5 — FBPerfMon
– Mon$ support
– TraceAPI support
31. 31 Conclusion
• To sleep well and live long you need:
– Relialble backups
– Continious monitoring and alerts
– Quick identification of problem
– Full details how to fix it
32. 32
Backups
Health
monitoring
Typical tasks
Indices maintenance
Transactions dynamics
SQL Queries
& plans
Database structure analysys
Database
