Tuesday, January 14, 2014

Reliability of Application Controls in ERP Systems Should be Questioned by Auditors

Reliability of Application Controls in ERP Systems Should be Questioned by Auditors
By Jeffrey T. Hare, CPA CIA CISA

The guidance related to relying on Automated (or Application) Controls was originally published by Institute of Internal Auditors (IIA) in July 2007. The IIA Global Technology Audit Guide number 8 (GTAG 8) takes into account the guidance provided by the PCAOB in Auditing Standard No. 12 (Appendix B, in particular).  The testing of an automated control typically requires the skills of both an internal auditor (functional or process auditor) as well as the skills of an IT auditor.

In fact, in GTAG 8, the IIA recommends that the help of an IT auditor is needed.[1]  IT auditors are typically needed to help test IT general controls such as controls over changes to the configurations, code changes, segregation of duties, and access controls.

The most common way to test the automated controls is through a “benchmark strategy.”  The IIA guidance summarizes the PCOAB standard as follows “If general controls that are used to monitor program changes, access to programs, and computer operations are effective and continue to be tested on a regular basis, the auditor can conclude that the application control is effective without having to repeat the previous year’s control test.  This is especially true if the auditor verifies that the application control has not changed since the auditor last tested the application control.”[2]

Since IT auditors are typically tasked with testing IT general controls, the IT auditor normally is responsible for testing ‘program changes, access to programs, and computer operations’ as well as ‘that the application control has not changed since the auditor last tested the application control.’

The beauty of ERP systems, such as Oracle’s E-Business Suite and SAP, is that an application control can be configured to meet an organization’s different requirements.  The ‘configurations’ drive the design of the control and, therefore, a change to the configuration can change the design of the control.  From a Sarbanes-Oxley perspective, the PCAOB and your external auditors should care immensely about whether or not the configurations related to key controls change from year to year.  If these configurations change, the auditor of the related application control should re-benchmark the control or test the control in another way.

The Problem

The problem faced by most organizations is there is no change history specifically built into the system for the configurations related to key controls.  In fact, as most auditors have discovered, little change history exists in most ERP systems unless the system is built to do so.  In ERP systems, such as Oracle’s E-Business Suite, the change history of a transaction only exists in a few cases and it rarely exists for configurations.
This leaves an IT auditor with a dilemma when trying to test the ‘general controls used to monitor program changes’ as is required in the IIA and PCAOB guidance.  How does an auditor get comfortable that no change has been made during an audit period? 

Absent audit history created by the system, ERP systems have built-in ‘row-who’ information.  This information typically tells you when a record is created and when it is last updated.  Therefore, an auditor can reasonably assume that when the ‘last updated by’ time stamp isn’t within the current audit period (and should be in the period in which the configuration was ‘baselined’ or earlier) that the configuration hasn’t been changed. 

There are three problems with relying on the ‘row who’ information:
  •  Most systems don’t have the queries predefined
  •  A false positive can be occur when data from another column is changed
  •  Some ‘back door’ methods of updating the data don’t change the ‘row who’ information

Lack of predefined queries

In an average system, there may be dozens of configurations related to key controls.  Ideally, the software provider would develop a ‘read only’ role that auditors could use to query the data.  Sadly, this information isn’t provided by most software companies or this information is only available in another product you need to license such as a GRC suite.

False positives

Another issue is false positives whereby a different column is updated than the column related to the key control.  Take a look at this change history:
Column A
Column B
Created Date
Updated Date
Initial data entry
Change 1
Change 2

In this example, column A’s data is NOT related to the key control and column B’s is what impacts the design of the key control.  As change 1 and change 2 are made, the updated date changes from 01-Jan-2014 to 15-Jan-2014 to 19-Jan-2014.   So if you are an auditor looking at the change history during 2014 you’d could falsely draw the conclusion that the configuration related to the key control (Column B) was changed when actually it was a different column (Column A) that was changed.

Back door methods of updating data

Finally, back door methods such as the use of an unsecured database login or SQL injection could change the data in column B, but not update the Updated Date column.  This risk is especially acute in certain systems such as Oracle E-Business Suite that provides over 50 forms and web pages that allow SQL injection (and in some cases ability to execute packages or OS scripts).

The Solution

If auditors are to rely on application controls and have reasonable assurance of operating effectiveness, a detailed audit trail / change tracking history is essential.  Most auditors give organizations a ‘pass’ when they don’t have detailed change history and take a sample from the change management tickets rather than insist the client develop the change history so the sample can be taken from the system.  We believe this approach CANNOT provide reasonable assurance related to the changes to configurations related to key controls.
The technology necessary to develop this change history differs from system to system.  In our area of domain expertise, Oracle E-Business Suite, there are reasonably priced third party systems that can be deployed using database triggers to create the necessary change history.  The benefits of such technology extends well beyond Sarbanes-Oxley ‘key control’ requirements and can be applied to updates to master data (such as supplier bank accounts) as well as other requirements (password resets for critical access accounts, forms that allow SQL injection, etc.).

The Conclusion

So here we are in 2014 – nearly seven years after the issuance of guidance by the IIA and the challenges facing reliance on automated controls are not much different than those faced when the guidance was first issued.  The time has come for auditors to hold client’s feet to the fire.  As it relates to application controls, either insist on a system-based audit trail or disqualify the reliance on the application control.  You’ve given a ‘pass’ to your clients for far too long when reasonable assurance cannot ‘reasonably’ be attained using the current approach.

About the Author

Jeffrey Hare, CPA CIA CISA is the founder and CEO of ERP Risk Advisors.  His extensive background includes public accounting (including Big 4 experience), industry, and Oracle Applications consulting experience.    Jeffrey has been working in the Oracle Applications space since 1998 with implementation, upgrade, and support experience.  Jeffrey is a Certified Public Accountant (CPA), a Certified Information Systems Auditor (CISA), and a Certified Internal Auditor (CIA).   He has worked in various countries including Australia, Austria, Canada, Mexico, Brazil, United Kingdom, Ireland, Saudi Arabia, and Germany.  Jeffrey is a graduate of Arizona State University and lives in northern Colorado with his wife and three daughters.

[1] IIA GTAG 8, page 14 “Need for Specialized Audit Resources”
[2] IIA GTAG 8, page 3 “Benchmarking”