The big data refinery (step 2 on image) is a new system capable of storing, aggregating and transforming a wide range of multi-structured raw data sources into usable formats that help fuel new insights for the business.
What does it have to do with IT audit? Everything. In fact, there are numerous ways for auditors to utilize the big data refinery. For instance, auditors may use it for analysis of thousands of daily logs generated by firewalls or other security appliance. It would be humanly impossible to manually to review all this logs and associated meaningful meaning and conclusion to the data in a timely manner. Another interesting use of data refinery could be in fraud analysis of large volumes of transactional data. In fact, big data refinery can flag specific transactions out of a large population of data to investigate for potential instances of fraud.
IT audit can benefit from emerging techniques, like big data refinery, that allow auditors to draw key conclusions from a wide range and large population of data sources. These conclusions may have an impact on the overall risk profile of companies.