An article describing the benefits of the 'shadow modelling' approach to review of financial models for project finance transactions. The importance of accounting and tax expertise is also highlighted.

1. Model Audit – there’s even more to it!
Jerome Brice, July 2009
A version of this article appeared on IJ online
Key terms: model audit, project finance, financial modelling, PFI, PPP
http://www.mazars.co.uk/projectfinance
jerome.brice@mazars.co.uk
I read with great interest Andy Hucknall’s recent article ‘Model Audit – do you know what
risks you are taking on’ which appeared on these pages. I very much welcome a debate
on how to best obtain risk assurance from your model auditor and share his concerns that
the standard of model audit service can be at best variable. However, while I agree with
his observations on the symptoms of poor model audit practice and indeed to the spirit of
his suggested solutions – I do take issue with three key elements of his diagnosis.
Firstly, I do not believe that the ‘inspection’ methodology he advocates is superior
to ‘shadow modelling’ ; instead I believe that shadow modelling as we at Mazars
perform it is a more robust process that provides the end-to-end test of model
logic that cell inspection cannot.
Secondly, Andy advocates a ‘risk rather than price’ based approach and believes
a ‘full audit’ should always be undertaken. I believe this is too simplistic; as every
project financier knows risk and price go together. What is needed at the outset of
a model audit is an informed discussion between auditor and client on the scope
of work and a shared understanding of the assurance this will provide.
Finally, Andy states that a model audit should be ‘assessed’ by ‘senior,
experienced staff’ and mentions that his team includes ‘qualified accountants’ with
a ‘corporate finance’ background. I would go further, model audits should be
‘undertaken’ by experienced staff with a ‘project finance’ background, the work
should be ‘assessed’ at Partner level. The team should also have full access to
technical teams who can give informed opinions on the latest developments in tax
legislation and accounting standards.
‘Shadow modelling’ versus ‘inspection’
The first misconception I would like to correct is that shadow modelling is limited
to the ‘top down’ elements of a full scope review. This, at least as undertaken at
Mazars, is simply not the case. When we create a shadow model of the model
under test the task is to fully reconcile all of the model logic – therefore this
technique is just as granular or ‘cell-by-cell’ as the alternative ‘inspection’
approach.

2. The key advantage of shadow modelling is that if we use the same inputs as the
model under test we expect to get the same answer. If the answer in any cell
which impacts upon model outputs is different, we keep asking the question
‘Why?’ until either any error in the model under test is amended or we have a
complete explanation as to why differences arise. Contrast this with the inspection
approach where the auditor is comparing the model under test with a model in his
or her head – my experience is that the model in my head is easily influenced if I
am looking at the logic that someone else has created! Another key difference is
the audit trail. At the end of the shadow modelling process we have a shadow
model which fully reconciles any differences in the model under test – this forms a
full proof of model logic from inputs through to outputs. Contrast this with the
inspection approach – at the end of the process all that you have is a printed list of
formula with some ticks!
There is a further major disadvantage of the inspection approach - it is far too
boring! The human brain is not designed to cope with repetitive checking – and
errors will inevitably be missed. The shadow modelling process in contrast is an
intellectual challenge, and even if the shadow modeller should become bored –
the process doesn’t allow short cuts, the lack of proof of reconciliation is there for
all to see. This observation is borne out by the research; Prof Ray Panko (the
same expert Andy refers to) in his paper “What We Know About Spreadsheet
Errors” updated in May 2008 refers to survey evidence which suggests individuals
undertaking ‘code inspection’ can be expected to capture around 60% of
spreadsheet errors. This would surely leave you with sleepless nights considering
the 40% of errors that may be missed? While I do not claim that the shadow
modelling is a panacea for all spreadsheet errors I do believe it is by far the best
solution to the audit of project finance models.
‘Risk rather than Price’ or ‘Risk and Price’
In his article, Andy outlines a standard set of objectives for a model audit and
suggests that this should always be followed – he views proposed changes to this
set of objectives as an auditor ruse to ‘de-scope’ and save cost. These objectives
are indeed familiar as the scope of work often set out by our clients and is one
that we are happy to undertake. However, this does not mean that this scope of
work cannot be discussed. Our main clients are leading banks and equity
investors who are undoubtedly ‘informed’; bringing as they do experience of
multiple deals backed by a team of lawyers specialising in ensuring that risks are
properly allocated. As such, they are not naïve and cannot be hoodwinked into
accepting a de-scoped service they don’t want. However, what they do welcome is
a commercial discussion on the appropriate scope of work for the assurance they
require. For example, they may wish to omit the tax requirement if they have
retained separate tax advisors or alternatively, they may wish to reinforce areas
where they have project specific concerns. In such cases we would of course
have an informed discussion on the balance of risk versus price – this is a
sensible commercial approach from both client and auditor perspective - one size
does not fit all.

3. The ability to offer informed opinions
One of the areas where I do agree with Andy is in the importance of the
involvement of senior staff in the model audit process. There is a wider point here
and while I have dwelt on the importance of a robust process to model logic, my
experience is that this isn’t the major source of material error in models. The main
sources or error are inconsistency of the model with contract documentation and
non-compliance of the model with accounting standards and tax legislation.
My main recommendations for avoiding errors related to contract documentation
is to ensure that document review is undertaken by experienced staff who
combine an understanding of commercial imperatives at financial close with the
knowledge of the financial realities that the project company will face when
operational. The second is to ensure that a clear process is agreed to ensure that
a full suite of document is made available to the auditor prior to financial close. As
clients of Mazars will recognise we facilitate this process by ensuring that a
Partner or Director is fully involved and visible at all stages of the audit including
guaranteeing our attendance at financial close.
The other types of error which we see all too often in the operational environment
relate to accounting and tax. We frequently see models which have had to be
restated as the statutory auditor has not agreed the accounting treatment
employed at financial close and we have also been contacted by project
companies in difficulties because the tax assumptions have not held because they
were either too simplistic or plain wrong. On the basis of this experience, I do not
believe it will continue to be tenable for model auditors to sign off the ‘accounting’
and ‘tax’ if they do not have access to expert teams who fully understand the
latest developments in international accounting standards and tax legislation. It is
simply not enough to merely have ‘qualified accountants’ in the team or ‘access’ to
a tax expert. For example, the impact of the implementation of International
Financial Reporting Standards has far reaching consequences for infrastructure
companies. Such developments can only be properly assessed by technical
teams who liaise directly with the standard setters and who regularly work on
international GAAP on conversion projects. Similarly, as HM Revenue and
Customs becomes increasingly vigilant it is important concession companies have
the most up-to-date advice on both corporation and indirect taxes. This sort of
information cannot be the responsibility of one ‘go-to’ team member, it must reside
within fully resourced expert technical teams.
In concluding, I am aware that I have been robust in my responses to some of the
assertions that Andy makes – while I feel this is justified, I am happy to accept that
there may be different approaches that achieve the same objective of concrete
model assurance. In addition, there are certainly two important views that I am
sure Andy and I both share. The first is that model audits are very necessary;
material mistakes in models are common and whichever model auditor is
employed they are likely to add considerable comfort as to the deliverability of a
transaction. The second is the observation that the choice of model auditor is all
too often made solely on price. As the discussion above has demonstrated there

4. are key differences to the way that the leading model audit firms approach an
audit and the value they add – to quote Andy once more, in selecting you model
auditor ‘above all, you need to know what risk you are taking on’.