1) The application called "Project PAMI" aims to reduce door-to-balloon times in cardiac cath labs by improving the user interface with Kinect. It allows zooming, panning, and navigating CT scans through gestures as well as controlling C-Arm movement with voice commands. 2) This could benefit millions by reducing mortality from acute myocardial infarction. It may also reduce radiation exposure for cardiac cath lab staff and allow them to aid in diagnosis and treatment. 3) The use of gestures and voice recognition with Kinect to control medical imaging and equipment is novel and innovative. It has potential to significantly improve workflow compared to traditional remote controls.

1. IMAGINE CUP 2012
KINECT FUN LABS CHALLENGE
ROUND 1 PROJECT PLAN TEMPLATE
Kinect Fun Labs Challenge Project Plan for [Alternate Reality]
Instructions
This is the Kinect Fun Labs Challenge Round 1 Project Plan Template. This is your Team’s Round 1 Entry
Requirement. It is designed to guide you to include the required components of your Team’s project. Please
use the questions in each section to align your submission with the judging requirements. Steps:
1. Insert your team name above
2. Answer the questions below. Please be thorough.
3. This Application Summary must not exceed 15,000 characters including spaces.
4. The character count starts below the line identified here*.
5. This document must be submitted in the English Language.
6. This document must be named as follows:
Kinect_Fun_Labs_Challenge_Round_1_Project_Plan_[Team Name]. DOC, .DOCX or .PDF,
.RTF or .TXT.
Submit your Team’s Round 1 entry by utilizing the submission form on the entry panel of the Kinect Fun
Labs Challenge page at imaginecup.com no later than then closing date of Round 1 (6 March, 2012, 11:59
GMT).
Questions
1. What problem are you solving as it relates to the Imagine Cup Theme?
Describe the real world problem you are working to solve (not the application itself – that information goes
below). Who will benefit from having this problem solved? How will they benefit? Will your solution impact a
large number of people very broadly, or a smaller number of people very deeply?
2. Name and Description of your Application:
What is the name of your Application or Creation? Describe your Application in detail.
3. Originality & Innovation:
How unique and original is your idea? Is the technology itself new and innovative, or is it the application of
existing NUI technology that is compelling? Were you inspired by an existing application of the Kinect?
4. Pre-existing source code:
If pre-existing source code files or third-party binary libraries are to be incorporated into the Application or
Creation (such as physics and game engines or control toolkits), then this pre-existing source code must be

2. clearly identified below and must not infringe on any third party rights, and must be used in accordance
with all applicable licensing and use terms. This includes images, music and source code. Tell us what open
source you are using, who owns it and what the applicable license information is.
5. Unique NUI Features:
Describe the Natural User Interface (NUI) features of your application. You must use features that are readily
available in any version of the official Microsoft Kinect for Windows SDK. At this time, you should have a
good idea of any supporting back-end infrastructure or cloud resources that your application requires. If your
application uses any cloud-based or other remote services, please describe the back-end infrastructure in
detail. Submissions that demonstrate technical innovation beyond these baseline features will be highly
regarded.
6. Usability:
User interface applications need to be as intuitive as possible. Is the application easy to use, or does it require
extensive training or trial-and-error? Does the application provide hints or other indicators to show the user
how to interact?
7. Presentation & Polish:
If you were to make a video about your project, what key points would you highlight? What scenarios would
you show the viewer to clarify the purpose and impact of your project?
*Character count starts below this line. Please remember that any application summary that includes more
than 15,000 characters, including spaces, will be disqualified. We strongly recommend that you confirm
your character count prior to submission.

3. 1.) Description of Problem and Beneficiaries:
Acute Myocardial Infarction is one of the most widely cited causes of death among
millions in the world.
Statistics for the same are available at the end of this section. The statistics are far worse
for developing countries like India, Pakistan, China, and Thailand.
By using our gadget to improve the user interface of a cardiac cath-lab, we try to
reduce these mortality rates indirectly by allowing hospitals around the world to better utilize
their resources to provide better health-care to millions.
„‟We also reduce the „door-to-balloon‟ time or the time duration of the procedure that is
critical in order to significantly reduce the mortality rate due to acute MI (Myocardial
Infarction).‟‟
Cardiac Cath-lab
RCIS(Registered
Cardiovascular
Invasive Specialist)
Performs Required
invasive procedures
RT(Radiologist CVT(Cardiovascular CCU attending +ICU
RN(Registered Nurse) Another CVT/RN Optional Rookie
Technician) Technologist) Resident
Controls C-Arm Controls and
Documentation of Optionally required in
Gives anesthesia and movements as well as monitors video feed Generally guided by
Diagnosis and CC(Critical Care
monitors its dosage Patient Bed at directions from the RCIS for training
Treatment Cases)
movements RCIS
The above diagram is the general breakup of a cardiac cath-lab staff. The (red) marks the
optional staff required for the invasive procedure while the (blue) marks the necessary staff
required for a procedure.
Typically, the people employed at a cardiac cath lab are exposed to radiation during the
course of a procedure. Despite common preventive measures, several leading cardiologists in
India have been diagnosed with cancer. Cardiac care staff in the cardiac cath-lab are sufficiently
susceptible to radiation as well, even more so because they generally have longer shifts in
comparison to the doctors.1
Our implementation using KINECT reduces the exposure risk for all three categories of
staff in the cardiac cath lab helping devote greater resources to analysis of the CT obtained,
while maintaining the health of the cath-lab staff.

4. Common scenarios that occur when a RCIS needs to analyze a particular frame are:
(i) He requests the CVT (Cardiovascular Technologist) to zoom the „image‟ frame. The CVT
moves into the console room (wasting valuable door-to-balloon time) and zooms the image
followed by panning the zoomed image to the required part.
(ii) The technician has a delay in hearing/listening to the request of the RCIS, responds late, and
skips a few frames ahead. This results in wastage of time since the technician has to trace-back
the sequence frame-by-frame.
(iii) Due to non-central position of the technician in comparison to the „wireless sensor‟
available, the „remote‟ signal emitted does not sufficiently reach the „wireless sensor‟ making
navigation through the CT feed significantly more difficult.
a) Large scale (Benefits for the common man) : Reduce patient mortality rates by
reducing the „door-to-balloon‟ time by improving the user interface of the C-Arm and the video
control.
b) Medium Scale: Reduce radiation exposure for cardiac cath-lab staff, while improving their
utilization in diagnosis and treatment of diseases. More importantly, if the cath-lab staff isn‟t
required to control the video-feeds or the C-Arm movements in the cardiac cath-lab, then they
could possibly aid in greater amounts towards the diagnosis and treatment of the patient or in
other parts of the hospital.
c) Small scale: It would also benefit doctors by reducing their exposure time per invasive
procedure.
[1]
http://www.theheart.org/article/1361685.do
“Coronary heart disease caused 1 of every 6 deaths in the United States in 2007. Coronary heart
disease mortality in 2007 was 406,351. Each year, an estimated 785 000 Americans will have a
new coronary attack, and 470,000 will have a recurrent attack. It is estimated that an additional
195,000 silent first myocardial infarctions occur each year. Approximately every 25 seconds, an
American will have a coronary event, and approximately every minute, someone will die of
one.”
A statistic obtained from ‘http://circ.ahajournals.org/content/123/4/e18.full.pdf’
The Create Study shows 61% of patients are admitted with STEMI in India.About 9.5 million
deaths, which is about one in six deaths worldwide, occur in the country every year. 2.37 million
people die of cardiovascular disease in India
A statistic obtained from Treatment and outcomes of acute coronary syndromes in
India (CREATE): a prospective analysis of registry data.
„The D2B Alliance advocates six key evidence-based strategies and one optional strategy to help
reduce door-to-balloon times:

5. 1. Cath lab team is available within 20–30 minutes (which will not be required as much
since the only people who need to be available after implementation are the RCIS and
the RN.)‟
A list of important procedures required to reduce door-to-balloon times from:
http://en.wikipedia.org/wiki/Door-to-balloon#Improving_door-to-balloon_times
2.) Name and Description of Application:
Our application is named: „Project PAMI‟. This stands for „Primary Acute Myocardial
Infarction‟. Primary Acute Myocardial Infarction accounts for the greatest percentage of deaths
among cardiac-disease related deaths and can be significantly reduced by reducing the „door-to-
balloon time‟.
Application Description:
Our application deals with improvement in two particular areas of the cath-lab system,
replacing the normal Remote-Based User Interface with NUI and therefore, significantly
reducing the „door-to-balloon‟ time.
a) The User Interface for Video Analysis: The current video-analysis tools in a modern
cardiac cath-lab are significantly disorganized and „time-consuming‟. Several factors attribute to
this:
(i) The zoom in and scroll image tools are not readily available at the remote. For an analysis, an
additional person is required to work from the console, which is again generally in a separate
room. This often results in waste-age of „door-to-balloon‟ time. We plan to improve this by
providing two gesture recognize-able functions:
When both the hands are brought closer it‟s a zoom in and when they are taken away it‟s a
zoom out gesture. This way the physician can zoom in/out a particular frame in the scan to
observe the defects more carefully.
 In addition, a cursor is provided after the zoom in gesture to scroll around the zoomed
segment. This cursor can be used for scrolling by tracking the movement of the using a „point‟
gesture. The direction point will cause the scrolling to occur at a pre-determined speed which can
be changed by the physician if required.
(ii) Standard controls that are provided on the remote are not sufficiently robust. The time delay
between the „button-press‟ and the „acknowledgement‟ on screen is significant. This causes
further increase in „door-to-balloon‟ time for a procedure, especially if the desired frame is
missed during the course of the procedure.

6. We plan to make it more natural by providing a push gesture, point gesture and a swipe-left,
swipe-right gesture in order to navigate through the frames obtained.
(b) User interface for C-Arm movement: C-Arm movement is in general done by a
radiologist technician. Most of the C-Arm movement is highly repetitive for a common
procedure. There are 5 common „C-Arm Angiographic views‟. We plan to give activate voice
recognition on basis of a particular gesture in order to navigate to a particular view.
For more clarification: Perform Gesture A  Voice Recognition is activated  „Request „RAO
20 Caud 20‟ is a sample procedure someone using our gadget would have to do.
The five common angiographic views are:4
i) „RAO 20,Caud 20‟
ii) „PA 0, Caud 30‟
iii) „LAO 50, Caud 30‟
iv) „LAO 50, Cran 30‟
v) „PA 0, Cran 40‟
Each of these views will be enumerated with a voice command, and Voice Recognition will be
activated using a gesture command.
[4] http://www.askdrwiki.com/mediawiki/index.php?title=Coronary_Angiography
3.) Originality and Innovation:
The C-Arm movement based on Voice Recognition while controlling the Voice
Recognition via simple gestures is original for this field.
This is possibly the first time the CT-scan and catheterization lab have been influenced
significantly by the use of Kinect‟s revolutionary NUI interface.

7. 4.) Pre-existing Source Code:
A brief overview of the control toolkit for Philips system is as follows:
System Control
Physical Interface Blocks Software Architecture
•Geometry block •User Interface Layer
•X-Ray Generation Block •Application Layer
•Image Detection Block •Technical Layer
•Image Processing Block •Embedded Software Layer
•Image Display Block
•Image Storage Block
Application area: The User Interface Layer: This layer contains the software for the GUI
(the software interface on the data monitor and the Xper module) and NGUI (Geometry and
Review modules, view pads, etc). It translates user actions to elements (commands) of the
interface provided by the application layer. Furthermore, it provides information to the user
about the system state, based on state information provided on the interface of the application
layer. The UI layer is notified about changes in the application state by means of events
generated by the application layer. The UI layer is where a majority of programming tools and
interfaces are readily available for those who want to develop applications for Philips C-Arm.
This layer‟s access is sufficient for our requirements and its documentation is readily available in
two forms as open-source.
Source: http://docweb.khk.be/Patrick%20Colleman/ARM7/lpc-ARM-book_srn.pdf
Development toolkit is provided by Philips to work on an imaging module for its C-Arm. This
forms a part of interacting with the Philips microprocessor.
Source : 4522 981 37032 CSIP level 1 2-21 of 26DMR100835, Rev:01
Forms the software-hardware interaction module for Philips C-Arm, from „technician‟
booklet.(Freely available)

8. 5.) Unique NUI Features:
We intend to design a multimodal system (speech + gesture) using the following NUI features in
our application.
i) Skeleton tracking:
We use the skeleton tracker of Microsoft Kinect SDK to track few vital skeletal joints of the
doctor performing the CT scan. We plan to track right hand to use it as a mouse control. The
position of the right hand is shown on the screen and the doctor can move his hand to select
predefined control buttons that are displayed on the screen.
ii) Push:
When the right hand is moved slightly in the direction of depth (i.e,z axis) a push is detected.
When a push action is performed on any one of the predefined buttons displayed on the screen,
the action is taken accordingly. For example, there are two buttons displayed on the screen: 1)
Start and 2) Stop. The doctor can move his hand to focus the mouse pointer on the start/stop
button and perform a push action. This will start/stop recording the CT Scan.
iii) Swipe right/Swipe left:
When the hand is moved swiftly to left/right a swipe left/right is detected. This gesture can be
used to navigate through the frames back and forth.
iv) Zoom-In and Zoom-Out:
When both the hands are brought closer it‟s a zoom out and when they are taken away it‟s a
zoom in gesture. This way the physician can zoom in/out a particular frame in the scan to
observe the defects more carefully. In addition, a cursor is provided after the zoom in gesture to
scroll around the zoomed segment.
v) Audio recognition:
This is the secondary mode of interaction, gesture being the first. The doctor will be able to
perform various actions on the CT recording just by calling out the appropriate word/sentence
like for ex “delete from start to this” will delete all the frames from the beginning to the current
one.

9. 6. Usability
The application should be very intuitive and easy to use, since, we want to minimize the
„door-to-balloon‟ time. All of the features mentioned in the NUI section, are simple to use and
very intuitive.
On consultation with doctors at Krishna Institute of Medical Sciences, while proposing
this gadget, their thoughts were that it would greatly simplify things. The gestures are simple to
use and would be recognizing a single person only, by taking advantage of skeletal tracking
provided by Kinect. The C-Arm movement would be easy to perform with Voice Recognition
stepping in only when actually required.
A one week training period might be required however, to let the physicians get
accustomed to the changes in the interface.
7.) Presentation and Polish:
Some of the things which we‟d like to stress upon for our presentation would be:
a) Case studies showing the importance of door-to-balloon time and their influence on patient
mortality rates.
(i) „Using a multivariate logistic regression model, the adjusted odds of in-hospital mortality did
not increase significantly with increasing delay from MI symptom onset to first balloon inflation.
However, for door-to-balloon time (median time 1 hour 56 minutes), the adjusted odds of
mortality were significantly increased by 41% to 62% for patients with door-to-balloon times
longer than 2 hours.‟
Source: http://jama.ama-assn.org/content/283/22/2941.full
b) Possible improvements in door-to-balloon time because of „our gadget‟.
(i) In multivariate analysis, six strategies were significantly associated with a faster door-to-
balloon time. These strategies included having emergency medicine physicians activate the
catheterization laboratory (mean reduction in door-to-balloon time, 8.2 minutes), having a single
call to a central page operator activate the laboratory (13.8 minutes), having the emergency
department activate the catheterization laboratory while the patient is en route to the hospital
(15.4 minutes), expecting staff to arrive in the catheterization laboratory within 20 minutes after
being paged (vs. >30 minutes) (19.3 minutes), having an attending cardiologist always on site
(14.6 minutes), and having staff in the emergency department and the catheterization laboratory

10. use real-time data feedback (8.6 minutes). Despite the effectiveness of these strategies, only a
minority of hospitals surveyed were using them.
Source: http://www.nejm.org/doi/full/10.1056/NEJMsa063117
One of these methods namely, „expecting staff to arrive in the catheterization laboratory 20
minutes after being paged‟ will be made much easier, since with lesser requirement of staff,
lesser mean delay in arrival of staff can be expected. More detailed case studies dealing with the
same would be appropriate as well.
c) Case studies showing the effects of radiation in the catheterization lab in hospitals.
„Cancer risk from professional exposure in staff working in cardiac catheterization laboratory:
Insights from the National Research Council's Biological Effects of Ionizing Radiation VII
Report.‟
„Conclusions: Cumulative professional radiological exposure is associated with a non-negligible
Lifetime attributable risk of cancer for the most exposed contemporary cardiac catheterization
laboratory staff.‟
Source: http://wrp-usa.com/images/Radiation_cardiac_catheterisation.pdf