A few thoughts on real world throttling

1. While writing my previous blog https://prashanthpanduranga.wordpress.com/2015/04/22/architecting-
extremely-large-scale-web-applications-a-must-read-for-every-architect/,I hadplannedatrip to Tirupati.
It was long due and my LONG trip away from home to Liverpool was scheduled to be a week after.
My lastcoupleof visitswaswith friends andthiswasthe first visitwithChaitra (Wife) andRynia(my princess
daughter), and secondvisit with my mom and dad. The first with them was a long time ago when I was a
little boy. So I was super excited.
What I did NOT realize was that, it was a Friday, with Lunar Eclipse the next day and the summer holidays
had just started. On a Lunar Eclipse the temple was scheduled to be closed for over 11 hours. Being the
most visited place of worship in the world, the usually 100,000 visitors a day seemingly was double that.
It is usually good to book trips with agencies who organize the complete package. They obviously know
what’sbest? NOTREALLY.... Tirumala now has booking online, which obviously I dintknow, theagency dint
recommend thateither. Whenwe gotthere, they saidthe bestandthefastestway isvia theSarvaDarshan,
the free darshan, without any pre-booking, that’s the only option. To top it all, the Chief Minister was
visitingthe temple andthatonly meanslonger waiting time. . . . Wereached Tirupatiat3:30 am . . .and the
wait began . .
Waiting in the line, my bulwark against any kinds of phobia, had started to feel being crushed by
demophobia, 100’s of 1000’s of visitors rushing their way through like particles accelerated by a force
within, beingin alarge hadroncollider, I couldn’tbutstopthinkingabout thesheernumber, andthetemple
authority(TTD: Tirumala Tirupati Devasthanams) quandary.
Imagine processingthat many numberof requests - Cannotload balance, No denial of requests, Noerror
codes, requests cannot be processed concurrently, No way to distribute requests and No special way to
handle bursts in activity such as lunar eclipse
Few measures had been taken by TTD though: Throttling, Throttling and more Throttling

2. Queuing requests – Needless to say Sarva Darshan queue was the longest
There are different types of queues. Free queue (Sarva Darshan), Paid queues (multiple of them – All
booked via online now) and queue where visitor climb up the hills via the steps
And as we can expect, they are prioritized queues. Sarva Darshan having the lowest priority.
While bits and pieces of the queue processing design patterns such as Throttling pattern, Priority Queue
Pattern, Queue based load levelling pattern all seemed to have been applied here, It also reminded me of
the Leaky bucketalgorithm, where visitorscome tothe queuelike a water being addedto the bucket, with
a maximum request quota of few thousands beyond which is throttled with a restore rate of a hours.
Yes Hours !
Rynia kept the trip alive and fun with her playing, skillfully using the sketch pen around my neck and
Govinda, Govinda in between repeating after the piligrims
It is time to introduce the VaikuntamQueueComplex (VQC). A 57 room facility with 3 levels and19 rooms
on each level. After movingthroughthe standingqueue for hours, thevisitors are throttledin this facility.
We reached this facility by 12:30 pm. We were made to wait in it till 10:00 pm, before the queue started
to move again

3. When we came to thatroom, there was a boardwhich said thatthe darshanam (viewing God) is next day
morning4:00 am. In front of us, was a life size television with live abhishekamindisplay. Justlike
customersleaving website viewing somecached content, the wait time was so muchthat a few folks
decided to leave.
There was photocard systemin the buildingwhere in youcould get the cards, leave the buildingand get
back later, however apartfrom the “4:00 am” signthere wasn’tanybody whowould/couldexplain the
reasonfor the delay. Rememberno error codes, notime delay messages either.
And that“4:00 am” messagewas acting like a proxy server/request filter redirecting messagesto the CDN
with the cached requests, pushingvisitorsaway to come back to visit anotherday.
All credit to TTD thoughtoensure thatno visitorstayshungry. Be it the standingqueue or in the VQC,
frequently the team wouldcome by, to server food, snacksand all for free of cost.

4. The queue startedto move againat 10:30 pm. There was a small detour of further throttlingto collect
the ladduprasadamtickets anda confluence of visitors from all the queues. Finally an hourafter we
were endowed to be in front of the omnipresent.
Dint fail to notice one more thing, the request accelerators (TTD staff), who will not let u stand beyond a
second in front of the GOD
I have to say, the power of faith is far greater than mostperceive. Even after hoursand hoursof standing,
walking and waiting, My momwho has a nagging pain in her knees, dint feel the pain, and rather felt that
the pain she had, has reduced, my daughter got her wishful toys, my wife and dad happy and myself
relieved that the trip was safe, satisfied and venerational at different levels.
.