Reliability of Application Controls in ERP Systems Should be Questioned by
Auditors
By Jeffrey T. Hare, CPA CIA CISA
The guidance related to relying on
Automated (or Application) Controls was originally published by Institute of
Internal Auditors (IIA) in July 2007. The IIA Global Technology Audit Guide
number 8 (GTAG 8) takes into account the guidance provided by the PCAOB in
Auditing Standard No. 12 (Appendix B, in particular). The testing of an automated control typically
requires the skills of both an internal auditor (functional or process auditor)
as well as the skills of an IT auditor.
In fact, in GTAG 8, the IIA
recommends that the help of an IT auditor is needed.
IT auditors are typically needed to help test
IT general controls such as controls over changes to the configurations, code
changes, segregation of duties, and access controls.
The most common way to test the
automated controls is through a “benchmark strategy.” The IIA guidance summarizes the PCOAB
standard as follows “If general controls that are used to monitor program
changes, access to programs, and computer operations are effective and continue
to be tested on a regular basis, the auditor can conclude that the application
control is effective without having to repeat the previous year’s control test. This is especially true if the auditor
verifies that the application control has not changed since the auditor last
tested the application control.”
Since IT auditors are typically
tasked with testing IT general controls, the IT auditor normally is responsible
for testing ‘program changes, access to programs, and computer operations’ as
well as ‘that the application control has not changed since the auditor last
tested the application control.’
The beauty of ERP systems, such as
Oracle’s E-Business Suite and SAP, is that an application control can be
configured to meet an organization’s different requirements. The ‘configurations’ drive the design of the
control and, therefore, a change to the configuration can change the design of
the control. From a Sarbanes-Oxley
perspective, the PCAOB and your external auditors should care immensely about
whether or not the configurations related to key controls change from year to
year. If these configurations change, the
auditor of the related application control should re-benchmark the control or
test the control in another way.
The Problem
The problem faced by most
organizations is there is no change history specifically built into the system
for the configurations related to key controls.
In fact, as most auditors have discovered, little change history exists
in most ERP systems unless the system is built to do so. In ERP systems, such as Oracle’s E-Business
Suite, the change history of a transaction only exists in a few cases and it
rarely exists for configurations.
This leaves an IT auditor with a
dilemma when trying to test the ‘general controls used to monitor program
changes’ as is required in the IIA and PCAOB guidance. How does an auditor get comfortable that no
change has been made during an audit period?
Absent audit history created by the
system, ERP systems have built-in ‘row-who’ information. This information typically tells you when a
record is created and when it is last updated.
Therefore, an auditor can reasonably assume that when the ‘last updated
by’ time stamp isn’t within the current audit period (and should be in the
period in which the configuration was ‘baselined’ or earlier) that the
configuration hasn’t been changed.
There are three problems with relying
on the ‘row who’ information:
- Most
systems don’t have the queries predefined
- A
false positive can be occur when data from another column is changed
- Some
‘back door’ methods of updating the data don’t change the ‘row who’ information
Lack of predefined queries
In an average system, there may be
dozens of configurations related to key controls. Ideally, the software provider would develop
a ‘read only’ role that auditors could use to query the data. Sadly, this information isn’t provided by
most software companies or this information is only available in another
product you need to license such as a GRC suite.
False positives
Another issue is false positives
whereby a different column is updated than the column related to the key
control. Take a look at this change
history:
|
Change
|
Column
A
|
Column
B
|
Created
Date
|
Updated
Date
|
|
Initial data entry
|
Yes
|
Yes
|
01-Jan-2014
|
01-Jan-2014
|
|
Change 1
|
No
|
Yes
|
01-Jan-2014
|
15-Jan-2014
|
|
Change 2
|
Yes
|
Yes
|
01-Jan-2014
|
19-Jan-2014
|
In this
example, column A’s data is NOT related to the key control and column B’s is
what impacts the design of the key control.
As change 1 and change 2 are made, the updated date changes from 01-Jan-2014
to 15-Jan-2014 to 19-Jan-2014. So if
you are an auditor looking at the change history during 2014 you’d could
falsely draw the conclusion that the configuration related to the key control
(Column B) was changed when actually it was a different column (Column A) that
was changed.
Back door methods of updating data
Finally, back door methods such as
the use of an unsecured database login or SQL injection could change the data
in column B, but not update the Updated Date column. This risk is especially acute in certain
systems such as Oracle E-Business Suite that provides over 50 forms and web
pages that allow SQL injection (and in some cases ability to execute packages
or OS scripts).
The Solution
If auditors are to rely on
application controls and have reasonable assurance of operating effectiveness,
a detailed audit trail / change tracking history is essential. Most auditors give organizations a ‘pass’ when
they don’t have detailed change history and take a sample from the change
management tickets rather than insist the client develop the change history so
the sample can be taken from the system.
We believe this approach CANNOT provide reasonable assurance related to
the changes to configurations related to key controls.
The technology necessary to develop
this change history differs from system to system. In our area of domain expertise, Oracle
E-Business Suite, there are reasonably priced third party systems that can be
deployed using database triggers to create the necessary change history. The benefits of such technology extends well
beyond Sarbanes-Oxley ‘key control’ requirements and can be applied to updates
to master data (such as supplier bank accounts) as well as other requirements
(password resets for critical access accounts, forms that allow SQL injection,
etc.).
The Conclusion
So here we are in 2014 – nearly
seven years after the issuance of guidance by the IIA and the challenges facing
reliance on automated controls are not much different than those faced when the
guidance was first issued. The time has
come for auditors to hold client’s feet to the fire. As it relates to application controls, either
insist on a system-based audit trail or disqualify the reliance on the
application control. You’ve given a
‘pass’ to your clients for far too long when reasonable assurance cannot
‘reasonably’ be attained using the current approach.
About the Author
Jeffrey Hare, CPA CIA CISA is the
founder and CEO of ERP Risk Advisors.
His extensive background includes public accounting (including Big 4
experience), industry, and Oracle Applications consulting experience. Jeffrey has been working in the Oracle Applications
space since 1998 with implementation, upgrade, and support experience. Jeffrey is a Certified Public Accountant
(CPA), a Certified Information Systems Auditor (CISA), and a Certified Internal
Auditor (CIA). He has worked in various
countries including Australia, Austria, Canada, Mexico, Brazil, United Kingdom,
Ireland, Saudi Arabia, and Germany.
Jeffrey is a graduate of Arizona State University and lives in northern
Colorado with his wife and three daughters.
