This document summarizes common database and application issues including: - SNAT exhaustion from too many outgoing connections and available ports - Blocking queries that prevent other operations from executing concurrently - Different index strategies like index seeks that improve query performance versus scans Potential solutions include: connection pooling, scaling out app instances, using private endpoints, and modifying queries, transactions and indexes to reduce blocking.

2. Denis Reznik
• Data Architect
Intapp
/denis-reznik-2345023
@denisreznik
/denis.reznik.5
Sergii Bielskyi
Head of R&D
ELEKS
sergii-bielskyi-884a7a2b
@sergey_belskiy
/sergey.belskiy

3. Agenda
• Application Issue 1
• Database Issue 1
• Application Issue 2
• Database Issue 2
• Application Issue 3

4. ASYNC_NETWORK_IO Wait
0
Schedulers
Runnable Queue
1
2
3
1
Suspended Queue
1
SELECT * FROM Users
1
ASYNC_NETWORK_IO
ASYNC_NETWORK_IO
1
Client
Id Name
1 John Snow
2 Darth Vader
3 Peter Parker
4 Andriy Yarmolenko
Users
var command = new SqlCommand("SELECT * FROM Users", connection);
connection.Open();
var reader = command.ExecuteReader();
while (reader.Read())
{
// some business logic
}

5. Index Seek
…
…
1 .. 1M
1 .. 2K 2K+1 .. 4K 1M-2K .. 1M
1 .. 300 301 .. 800 801 .. 1,5K 1,5K+1 .. 2K
SELECT * FROM Users
WHERE Id = 523

6. Index Scan
…
…
1 .. 1M
1 .. 2K 2K+1 .. 4K 1M-2K .. 1M
1 .. 300 301 .. 800 801 .. 1,5K 1,5K+1 .. 2K
SELECT * FROM Users

7. SNAT
Source Network Address
Translation

8. SNAT exhaustion
Imagine having multiple browsers going to https://www.microsoft.com,
which is:
• Destination IP = 23.53.254.142
• Destination Port = 443
• Protocol = TCP
TCP SNAT port release
• If either server/client sends FINACK, SNAT port will be released after 240 seconds.
• If a RST is seen, SNAT port will be released after 15 seconds.
• If idle timeout has been reached, port is released.
65536 ports per load balancer * 5 / 2000 instances = ~ 160 SNAT ports per instance

9. Symptoms SNAT exhaustion
• Slow response times on all or some of the instances in a service plan.
• Intermittent 5xx or Bad Gateway errors
• Timeout error messages
• Could not connect to external endpoints (like SQLDB, Service Fabric,
other App services etc.)

10. How to solve that issue
From application:
• Modify the application to reuse connections
• Modify the application to use connection pooling
• Modify the application to use less aggressive retry logic
• Use keepalives to reset the outbound idle timeout
From database:
• DbContext pooling
From App Service:
• Scale out the App Service plan to more instances
• Use App Service Environment, whose worker instance can have more SNAT ports, due to its smaller instances pool size.
• connection pools: By pooling your connections, you avoid opening new network connections for calls to the same address
and port.
• service endpoints: You don't have a SNAT port restriction to the services secured with service endpoints. (VNet)
• private endpoints: You don't have a SNAT port restriction to services secured with private endpoints. (VNet)
• NAT Gateway: With a NAT Gateway, you have 64k outbound SNAT ports that are usable by the resources sending traffic
through it.

11. Sample
• DBContextPool
• Reuse HttpClient
• Release HttpWebRequest
• Scale out App service

12. LCK_M_* Waits
0
Schedulers
Runnable Queue
1
2 1
Suspended Queue
2
SELECT * FROM Users
WHERE Name LIKE 'K%'
UPDATE Users
SET Name = 'Jesse Pinkman'
WHERE Id = 3
Id Name
1 John Snow
2 Peter Partner
3 Kirill Nenahov
4 Jane Dow
1
Users
2
3
2
Id Name
1 John Snow
2 Peter Partner
3 Jesse Pinkman
4 Jane Dow
LCK_M_X
LCK_M_X
2
X S

13. Summary
• Paging
• Indexes
• SNAT
• Blocking
• Transaction

14. Denis Reznik
Data Architect
Intapp
/denis-reznik-2345023
@denisreznik
/denis.reznik.5
Sergii Bielskyi
Head of R&D
ELEKS
/sergii-bielskyi-884a7a2b
@sergey_belskiy
/sergey.belskiy

15. As an encore
…

16. Regular mistakes of using ORM (EF)
• Too greedy with Columns
• Too greedy with Rows
• Bulk insert
• Mismatched data types
• Autochangetracker
• AsNoTracking
• Dynamic types of arrays
• toList(), toGroup(), etc