Just finished Malcolm Gladwell's "What the Dog Saw" on a long plane ride this weekend. Like his other books (Tipping Point, Blink, and Outliers), there are great stories and examples for those of us involved in data analysis, including internal auditing and especially continuous auditing.
Gladwell's current book is actually a collection of essays from New Yorker magazine, but they piece together nicely so the essays can be read in sequence or by selecting chapters of interest. If you have time to only read one chapter, I'd point you toward the chapter "Open Secrets. Enron, Intelligence, and the Perils of Too Much Information."
The book points out that Enron's Special Purpose Entities (SPE's) were entirely transparent. To a fault. Because each of the more than 3000 SPE's involved paperwork of an average of 1000 pages of filings. Even an executive summary of an SPE contained 40 single-spaced pages. So the challenge in understanding the financial risks of their SPE's was to understand how to filter an insanely large volume of data into a form that was manageable, comprehensible, and actionable.
As you progress on the Continuous Auditing and Continuous Monitoring Maturity Curve, you'll find that your teams are amassing a similarly overwhelming (though hopefully not as large!) set of source data and anomalies to review. How do you see the source data? How do you see the exceptions? How do you decide which ones to act on?
Most data analysis efforts that we have worked have a goal of identifying individual exceptions, or rows, in database speak. So a AP vendor shares an address or tax ID number with an employee. Or a sales invoice had a discount in excess of a contract maximum. To act, we send an email to someone, maybe with a spreadsheet attached, to research and resolve the exception row.
But let's learn from Enron's SPE's. If we send 3000 emails, how will we manage the follow-up. Can we use color and graphs to measure the magnitude of the exceptions in total? How should we identify transactions that are acceptable one-by-one (example: a $9,500 requisition from a manager with a $10,000 signing authority), but unacceptable as a larger series (say ten, $9,500 requsitions from that same manager, all within the same week)?
Monday, February 21, 2011
Monday, January 31, 2011
How to update the IIA GTAG for what's new in Continuous Auditing?
Today's blog seeks to assimilate some of the things we heard at the IIA International conference last summer in Atlanta, in advance of this week's IIA Working Group discussions regarding the Global Technology Audit Guide (GTAG) on Continuous Auditing #3. The purpose of the Working Group discussions are to identify areas of the GTAG that require updating, so we are pleased to be able to participate with such an esteemed group of colleagues.
I wrote last summer about the IIA International conference, and how Data Analysis and Continuous Auditing were discussed at that three of the more interesting presentations at that conference. Those presenters Dan Kneer, Steve Biskie (ACL Services) and Robert Mainardi, and each presenter spoke on some combination of Data Analysis, Continuous Auditing, and Continuous Monitoring. Though they used many of the same words and terms, their perspective often seemed quite different.
Is Continuous Auditing about audit project selection and Risk Assessment. Yes. So techniques such as regression analysis, ratio analysis, and other analysis of aggregate data should be considered in any GTAG update. But Continuous Auditing is also about more frequent updates of subjective data, like control self-assessment, surveys, and call program activities. Similarly Continuous Auditing can and should include data analysis of transaction details. And greater frequency of data analysis, to include Visual Reporting also aid greatly in Risk and Control Assessment activity. Perhaps text analytics and analysis of unstructured data too.
Looking forward to this week's Working Group, so we can find some distinctive words to distinguish the various types of Continuous Auditing activities for inclusion in any updates to the GTAG.
Since Continuous Auditing can
Joe Oringel
Visual Risk IQ
Charlotte, NC USA
I wrote last summer about the IIA International conference, and how Data Analysis and Continuous Auditing were discussed at that three of the more interesting presentations at that conference. Those presenters Dan Kneer, Steve Biskie (ACL Services) and Robert Mainardi, and each presenter spoke on some combination of Data Analysis, Continuous Auditing, and Continuous Monitoring. Though they used many of the same words and terms, their perspective often seemed quite different.
Is Continuous Auditing about audit project selection and Risk Assessment. Yes. So techniques such as regression analysis, ratio analysis, and other analysis of aggregate data should be considered in any GTAG update. But Continuous Auditing is also about more frequent updates of subjective data, like control self-assessment, surveys, and call program activities. Similarly Continuous Auditing can and should include data analysis of transaction details. And greater frequency of data analysis, to include Visual Reporting also aid greatly in Risk and Control Assessment activity. Perhaps text analytics and analysis of unstructured data too.
Looking forward to this week's Working Group, so we can find some distinctive words to distinguish the various types of Continuous Auditing activities for inclusion in any updates to the GTAG.
Since Continuous Auditing can
Joe Oringel
Visual Risk IQ
Charlotte, NC USA
Saturday, November 6, 2010
Highlights of Day 2 Rutgers WCAS
More case studies on Saturday than Friday - presenters have included Hewlett-Packard, Proctor & Gamble, IBM, and Siemens Financial, among others. Highlights from these presentations include:
Best wishes,
Joe Oringel
Visual Risk IQ
Newark NJ
- HP presented their use of monthly data extraction and a variety of CAAT-based and ERP query tools to interrogate transactions and logs. They evaluate a mix of configurable controls and transaction analysis to deliver a risk-based heat map that aids the audit team in project selection decisions. They've made excellent progress from prior years, and continue to be a leader in CA / CM, especially among SAP shops.
- P&G presented about their measurement around the business case for their CA / CM investments, which have focused primarily around order to cash (O2C). Their program's strengths are its workflow, in that audit uses "automated delivery of high quality controls tests results to the business." It's the evolution of having MANAGEMENT evaluate the test results (vs. internal audit) that was most noteworthy.
- IBM presented about their system that they call Enhanced Auditing with Technology, which is also focuses on O2C. They monitor more than 400 query test attributes (contrast w/ Siemens Financial, who monitors only 45!).
- Jason Gross of Siemens Financial presented their CCM program with considerable energy and enthusiasm. Jason and I had previously met at an IIA event during 2007, when he had been in Internal Audit. Interesting is that he has left audit and is now a direct report to the CFO at Siemens Financial. This option should be on the career path of most data-focused, audit professionals as it allows Jason and his team to have more responsibility for research and follow-up on CCM exceptions.
Best wishes,
Joe Oringel
Visual Risk IQ
Newark NJ
Friday, November 5, 2010
Top 10 Things that Go Wrong in a Continuous Auditing Project
Very nice summary from Patrick Taylor from Oversight Systems of their experiences from CA and CCM implementation. Patrick did a very good job of sharing examples and screen shots of how their tool are being configured to monitor both routine and non-routine transactions.
10. Compliance is the Lead
9. Your eyes are bigger than your stomach (you try to monitor everything)
8. Look through this report please (tedious, there's no bottom to the report)
7. Let's learn a specialized analysis language (instead of SQL)
6. Let's clean up the last two years of exceptions (10000++ exceptions. Yikes!)
5. Continuous Audit instead of Continuous Improvement (i.e. Use Reason Codes)
4. Don't know how to spell Vasarhelee, Vaserheyli, Vasarhellee, Vasarhelyi... (LOL!)
3. Only know how to Audit AP (other apps are
2. Bringing a knife to a gun fight. (re-testing what is already controlled by ERP)
1. Not Using Oversight (LOL x 2)
10. Compliance is the Lead
9. Your eyes are bigger than your stomach (you try to monitor everything)
8. Look through this report please (tedious, there's no bottom to the report)
7. Let's learn a specialized analysis language (instead of SQL)
6. Let's clean up the last two years of exceptions (10000++ exceptions. Yikes!)
5. Continuous Audit instead of Continuous Improvement (i.e. Use Reason Codes)
4. Don't know how to spell Vasarhelee, Vaserheyli, Vasarhellee, Vasarhelyi... (LOL!)
3. Only know how to Audit AP (other apps are
2. Bringing a knife to a gun fight. (re-testing what is already controlled by ERP)
1. Not Using Oversight (LOL x 2)
More from Rutgers WCAS. ACL, XBRL, and Caseware RCM (alphabet soup indeed!)
Excellent presentation by ACL's John Verver on their Data Analytics Capability model. The shout out regarding our similar CA Maturity model was much appreciated. ACL's efforts to chart the path from one-time, retrospective data analysis (i.e. Hindsight) to more frequent, even predictive data analytics (i.e. Foresight) is on-target.
Noteworthy is that their model doesn't necessarily advocate "continuous" as the desired frequency for data analysis, either for Internal Audit or for management-led monitoring efforts. The right frequency depends on the relative risk of the process and data that is being analyzed.
These slides aren't up yet on the Rutgers site (UPDATE - Link provided above), but I'll look to post a link when they're uploaded. Very good content here for building a simple path toward more frequent, data driven auditing and monitoring.
Following John was Eric Cohen from PwC who provided some excellent information on the state of tagging and XBRL as a technique for automated data acquisition. The ability to acquire external data (e.g. competitor financial results) and compare those results to our own results is an excellent management tool, and one that is now beginning to be realized.
Following Eric Cohen was Andrew Simpson. Andrew is the Chief Operating Officer, CaseWare RCM Inc., formerly SymSure Ltd. Andrew's slide on the cycle (aka "yo-yo") of control measurement and how greater frequency yields continuously improving controls. Though CaseWare is a relatively new entrant in the CA / CM space, they seem to have excellent potential. UPDATE - link to Andrew Simpson's slides, which are now posted at WCARS site:
Noteworthy is that their model doesn't necessarily advocate "continuous" as the desired frequency for data analysis, either for Internal Audit or for management-led monitoring efforts. The right frequency depends on the relative risk of the process and data that is being analyzed.
These slides aren't up yet on the Rutgers site (UPDATE - Link provided above), but I'll look to post a link when they're uploaded. Very good content here for building a simple path toward more frequent, data driven auditing and monitoring.
Following John was Eric Cohen from PwC who provided some excellent information on the state of tagging and XBRL as a technique for automated data acquisition. The ability to acquire external data (e.g. competitor financial results) and compare those results to our own results is an excellent management tool, and one that is now beginning to be realized.
Following Eric Cohen was Andrew Simpson. Andrew is the Chief Operating Officer, CaseWare RCM Inc., formerly SymSure Ltd. Andrew's slide on the cycle (aka "yo-yo") of control measurement and how greater frequency yields continuously improving controls. Though CaseWare is a relatively new entrant in the CA / CM space, they seem to have excellent potential. UPDATE - link to Andrew Simpson's slides, which are now posted at WCARS site:
Rutgers WCAS - Advancing Audit Analytics. Key learnings
Session Moderated by Trevor Stewart (Retired Partner, Deloitte)
Panelists:
Dr. Rod Brennan (Siemens - Risk & Internal Control Officer)
Mark Loizeaux (Deloitte - Assurance National Office)
Amy Pawlicki (AICPA - Business Reporting and XBRL)
Phil Wedemeyer (Grant Thornton, Assurance National Office)
Key Learnings:
Panelists:
Dr. Rod Brennan (Siemens - Risk & Internal Control Officer)
Mark Loizeaux (Deloitte - Assurance National Office)
Amy Pawlicki (AICPA - Business Reporting and XBRL)
Phil Wedemeyer (Grant Thornton, Assurance National Office)
Key Learnings:
- Now that the SOX windfall is over for the large accounting firms, external audit fees are returning to the trends of fixed price work. Hence, external auditing firms are strongly encouraging their clients to increase the use of CA and data analysis, so they can review those results and gain greater assurance in the same or less number of hours.
- Knowledge of data analytics varies widely among the audit teams at the largest audit firms. Even members of the most advanced engagement teams in the "best" offices work on very low-tech, (i.e. limited use of data analytics) audits in the same office.
- Despite internal controls emphasis by the auditing firms and auditing standards, nearly all signing external partners have a greater level of trust in Balance Sheets than other audit procedures.
- The PCAOB believes that external auditing is a standard, continuous process that must be followed. Departures from standard process should be documented in audit workpapers. Identifying anomalies and explaining them is an integral part of this process.
- PCAOB Auditing Standards have been expanded to include rigorous guidance on how to do a risk assessment. Data analytics should contribute to this risk assessment.
- One of the downsides (per the panelists - not IMHO!) of more rigorous analytics is that we are more inclined to find anomalies and errors in financial processes. Having to investigate and explain these anomalies can be very costly. Example: 10,000 exceptions in Travel and Entertainment Expense review cannot practically be investigated and explained.
- Most auditors don't like graphics as much as columns of numbers, yet their stamina for reviewing columns of numbers isn't good enough. Graphical tools to aid in the interpretation of data is an area of interest for the panelists. We at Visual Risk IQ (emphasis added - Visual is our first name!) agree.
- Data sources that can be used to aid in continuous assurance are not limited to financial statements or internal systems. External data sources and internal operational system are excellent sources for insights on business risk.
- Who wants a better audit? Management, or do they want less obtrusive audits? Regulators, or do they want fewer auditor-reported issues? Auditors, or do better audits cause problem during litigation? Maybe investors, but not for greater costs. And investors may not even understand the audits they get now.
- The issues of connecting financial statements to underlying business processes and recording of transactions is a limitation of the audit profession.
- Auditors of public companies need to understand materiality from an investor point of view. What do investors depend on to make their investment decisions? Materiality is not merely xx% of revenue or assets for all companies, especially if much of the market cap is based on future revenue or earnings, not historical results.
- Most losses in market cap relate to failure in strategic risk, not financial risk. So is the emphasis on continuous auditing of financial transactions a flawed model?
- Internal auditors are challenged to access data that is needed for audit analytics.
Live again from Rutgers - 21st World Continuous Auditing Symposium
Wow, has it been nearly four months since I've blogged? Good news is that my brevity is improving. For those of you that don't follow me on Twitter ( @VisualRiskIQ or www.Twitter.com/VisualRiskIQ ), I've been at least fairly busy reporting on Fraud, FCPA, and especially Higher Ed operational and compliance issues in the news that can be positively influenced by Continuous Auditing (CA) and Continuous Monitoring (CM) applications.
Our firm continues its implementation of CA and CM for a variety of corporate, higher ed, and non-governmental organizations, and we continue to see an uptick in investment in the still-emerging technology. With that said, it's slow and cautious investment, at least in part because the return on these investments can be mixed, especially if they are seen as technology purchases and not fuller, solution-focused change initiatives that involve people, process, and technology.
The Rutgers Conference is a confluence of academia, external auditors, software firms, and internal audit customers of data analytics, so it is a very interesting venue. I look forward to documenting some of the soundbytes and lessons learned for folks who have not been fortunate enough to attend. For those in attendance, I welcome any comments or corrections to the notes that I'll be taking.
Regards,
Joe Oringel
Visual Risk IQ
reporting from Rutgers Business School
Newark NJ
Our firm continues its implementation of CA and CM for a variety of corporate, higher ed, and non-governmental organizations, and we continue to see an uptick in investment in the still-emerging technology. With that said, it's slow and cautious investment, at least in part because the return on these investments can be mixed, especially if they are seen as technology purchases and not fuller, solution-focused change initiatives that involve people, process, and technology.
The Rutgers Conference is a confluence of academia, external auditors, software firms, and internal audit customers of data analytics, so it is a very interesting venue. I look forward to documenting some of the soundbytes and lessons learned for folks who have not been fortunate enough to attend. For those in attendance, I welcome any comments or corrections to the notes that I'll be taking.
Regards,
Joe Oringel
Visual Risk IQ
reporting from Rutgers Business School
Newark NJ
Subscribe to:
Posts (Atom)
