This technical report summarizes three automation projects completed by the author during their co-op term at Powerex Corp: 1) ITSM/TFS Database Reporting automated the generation of weekly reports by querying two separate databases. 2) Prince & TxMS Latency Alert automated monitoring and alerting of latency between two applications by calculating time differences between database entries. 3) The automation process design follows a linear state transition through draft, development, testing, and production stages, with expected regressions forming a feedback loop to improve stability.

1. Engineering Technical
Report
Automation of Database Reporting
POWEREX CORP.
August 10, 2014
Jim Tu
Technical report submitted to Faculty of Applied Science in fulfillment of the co-op
requirement for the degree of
Systems Engineering

2. Page | 1
Abstract
The purpose of this report is to demonstrate the standard practice of automation of database
reporting and operational control during my co-op term in Powerex as IT Operations Analyst. In order
to fully illustrate the automation process, I will partition this report on the two related projects:
ITSM/TFS Database Reporting. ITSM/TFS Data reporting mainly focused on bridging the gap between
two separate databases system used by Powerex, I was tasked to build a process that gives more
versatility and controllability in the both databases for generating weekly reports. Prince & TxMS
Latency Alert is a typical example of alert automation that focused on detect scheduled jobs and how it
part take in the effort in maximizing existing resources . By structure each automation process to run
iteratively, it is more cost efficient, stable, and introduces less error compare to manual process.
To ensure the stability of the automation process, it is crucial to follow each stage of
development before deployment to production. In later part of this report I will summarize how each
three projects mentioned follows exact same design procedure, similar to a finite state machine where it
transitions between four linear states starting from draft, to development, to UAT testing, and finally to
production. However due to the UAT testing environment and production environment has slight
differentiation, it is observed that during the stage of development each process will have at least one
regression from production back to UAT. This phenomenon can be best explained as a feedback loop.
Where if discount any external influence to the development environments, it is assumed each
processes’ stability will be in positive feedback loop (more stable), and errors will be in a negative
feedback loop(less errors), and eventually stays in production environment permanently.

3. Page | 2
Table of Contents
Abstract ........................................................................................................................................................ 1
Table of Contents ......................................................................................................................................... 2
Introduction of Powerex ............................................................................................................................ 3
ITSM/TFS Database Reporting .................................................................................................................... 3
Prince & TxMS Latency Alert........................................................................................................................ 5
Automate Process Design ............................................................................................................................ 6
Conclusion .................................................................................................................................................... 6

4. Page | 3
Introduction of Powerex
Powerex Corporation is a subsidiary company of BC Hydro, responsible for marketing BC Hydro’s
surplus electricity in western North America. [2] There are plenty of customized application build in
house by developers in Powerex, two major front office applications being Prince and TxMS that
communicate bi-directionally to handles energy trade deals and transmission. IT support and IT
developers works collaboratively to ensure the front office business and back office has the tools they
need, and it is maintained and fully operational. The roles as IT Operations Analyst is responsible for
managing these scheduled jobs, maximize existing resources, and maintain system availability [1].
ITSM/TFS Database Reporting
Within the IT Operations and Infrastructure Team, there are four tier level of support. [Figure
1][Table 1] Every incident or request from the business will be logged in ITSM database by Tier1/2
DLS(Desktop LAN Support) for Incident Management. If an incident is logged are require further
troubleshoot it will escalate to Tier3 Operations, in which a ticket will be logged in TFS database for
Problem Management. If turns out it is an application issue, and requires outages, it will require a ticket
in ITSM for Change Management. For Tier4 developers to keep track of their work item, they use TFS
database to keep track of the tickets for Application Management. If an incident or problem is resolved,
each tier lead will the user who requested the incident/problem that the issue has been resolved.
Tier Structure Management / Database Used
Tier 4 Application
Support
Application
Management
TFS
Tier 3 Operations
Support
Change
Management
ITSM
Tier 2 Desktop LAN
Support
Problem
Management
TFS
Tier 1 Desktop LAN
Support
Incident
Managment
ITSM
Table 1.Tier Structure and Corresponding Managment/Database Table
Figure 1. IT Tier structure Explained

5. Page | 4
Due to the fact Powerex is direct subsidiary of BC Hydro, all the logged ticket from Change
Management and Incident Management are monitored externally by BC Hydro, and vendor that manage
the software for ITSM do not allow external query to its database. TFS database however is managed
internally by Powerex, and employee has full accessibility to the database.
So the proposed plan was to build clone version of ITSM database that Powerex can manage
internally, with the read access that downloads data from ITSM Clone and outputs into the reports for
Incident management and Change Management. [Figure 2].
POWEREX INTERNAL
ITSM- SQL
Query
TFS – SQL
Query
ITSM ITSM
Clone
TFS
Reports for
Incident Mgmt
Change Mgmt
Reports for
Problem Mgmt
Application Mgmt
Figure 2. Schematic for bridging the gap between ITSM and TFS database.
The benefit of having able to read into both databases gives IT team clearer visibility of the
tickets logged by each management level. By pulling data from database, an automated email reminder
was created to notify members of Powerex IT team weekly, which they can anticipate and plan
accordingly with the upcoming incidents, problems, and changes updated in real-time. [Figure 3]
Figure 3. An example template of the reminder email sent out automatically for Problem Managment.

6. Page | 5
Rather than just a plain flat table, it is also possible to automate graphs from querying into SQL
database. By downloading historic date from the database, the team is able to have graphical
representation of the entire year worth of ticket items. From using Tableau graphically analysis, reports
can now be pulled directly from database in real-time automatically. [Figure 4]
This project has greatly benefited the IT Infrastructure and Operations team in terms of
improves efficiency, accuracy and versatility of the reports for each management level. By introduce
automated process it excludes the repetitive manual labor that is time consuming, and ensure the
content accuracy is correct every time in real time. The content, format, and structure of the report can
changed easily, and also enable correlation between two different databases base on their
corresponding index. By looking at the historical trend, IT team was able to see the trend line month to
month and year to year, which can help the team allocate resources in advance base on the forecast.
Prince & TxMS Latency Alert
Prince and TxMS being the two major trading applications that are built in house by Powerex,
occasionally have latency issue between its communications with one another. When one application
sends a batch command to another, there are buffers build in to capture all the batches and put into the
queue. If one batch takes longer than usual to calculate, it will delay all other batches after it. The
Prince TxMS Latency Check queries into two different database, and calculates the time difference
between the when one application sends the batch, and when one application receives the batch.
[Figure 5] The challenging part of this alert is that calculation of latency has to be done outside of SQL
query, and calculate the send and received batch time in Automate BPA, a network automation tool.
Figure 4. Sneak peak of Ops Team Work Item Graph in Tableau

7. Page | 6
Figure 5. Prince and TxMS Latency Diagram.
The above diagram accurately identifies that it was communication between
TxMS.BatchSend.Time and Prince.BatchRecive.Time that were causing the deley in batch. An
automated email alert were created to notify any latency between TxMS back to Prince that are more
than 10s . Here is an example email template below:
Figure 6. Prince TxMS Latency Email Alert
With this finding the TxMS developer were able to parallel process the batches, and allow each batch to
be pipe lined to the other application, increase efficiency and minimized the transaction time.

8. Page | 7
Automate Process Design
When designing a automate process, there is certain stages of development has to be followed
to ensure the process does not interfere with the production environment. Starting from the draft,
where designer figure out the criteria to create the automate process. Then progress to next stage
where the drafted idea is created. Then the developed code is tested in UAT environment before it is
pushed to Production [Figure 7]. If it is stable in production, the process will state in production.
However it is expected that each automated process will at least fail once, this can be caused by
external influences such as database timeouts or server outage etc., but if it is not external influence the
stage for the automate process will be transition back to development for code change.
The design stages of the automate process is a finite state machine with linear state transition,
and the behavior of state regression are similar to a feedback loop, where each loop the automate
process goes through, ideally it would become more stable (positive feedback) with reduction in errors
(negative feedback). However sometimes it is difficult to identify if the automated process in
production failed due to its own code, or the external influence. Thus often times, designer will have to
iterated multiple times in feedback loop with net gain of zero.
Conclusion
Although an automated process of database reporting can be efficient, accurate, and versatile,
however if it becomes unstable at the production environment, it could be catastrophic to other
process running concurrent to this one. Therefore designer of the automated process will have to revert
back to development at least once, and by improve the process one feedback loop at the time until the
process can be standalone in production.
Figure 8. State Transitional Diagram for Automate Process Design

9. Page | 8
Glossary
Operations
Control
Executes day-to-day routine tasks related to the operation of infrastructure
components and applications. This includes job scheduling, backup and
restore activities, print and output management, and routine maintenance.
IT Operations
Analyst
Responsible for managing scheduled jobs, maximize existing resources,
and maintain system availability.
ITSM IT Service Management. Powerex uses Remedy IT Service Management
Suite by BMC Software, Which mainly used as a Service Management tool
to log incident and change tickets. These tickets are used for Incident
Management and Change Management.
TFS Team Foundation Server. A server based platform that also allow intra-web
interface. Powerex uses TFS as a tool to create and track Problem tickets,
and Developer tickets. These tickets are used for Problem Management,
and Application Management.
UAT User Acceptance Testing. A process of verifying that a system meets
mutually agreed-upon requirements.
Tableau Graphing software. It produces a family of interactive data
visualization products focused on business intelligence [3]
Automate BPA Network Automation, process automation software designed to
simplify repetitive information technology processes on desktop
computers and servers. [4]
Bibliography

10. Page | 9
[1] "What Does a Operations Analsyst Do?," wiseGeek, [Online]. Available:
http://www.wisegeek.com/what-does-a-operations-analyst-do.htm. [Accessed 10 8 2014].
[2] "About Us," Powerex, [Online]. Available: http://www2.powerex.com/AboutUs.aspx. [Accessed 10 8
2014].
[3] "Tableau Software," Wikipedia, 16 July 2014. [Online]. Available:
http://en.wikipedia.org/wiki/Tableau_Software. [Accessed 10 August 2014].
[4] "Network Automation," Wikipedia, 21 May 2013. [Online]. Available:
http://en.wikipedia.org/wiki/Network_Automation. [Accessed 10 August 2014].