In the current enterprise application landscape, most of the organizations prefer the leading BPM and case management vendors for their ability to automate and streamline business processes. Many of these vendor solutions collect the key process and interaction metrics and data to provide a clear insight into operations and to provide better work force management for the operations team. On the flip side, this data collected adds to data maintenance and management challenges. Many applications and solution providers will consider data management and retention only after the solutions age as well as accumulate enormous data related to process, metrics and interactions. Before the IT and the business team close on the approach to data retention and purging, the collected data would have already started to impact the applications.
I have seen this pattern across most of the organizations that we have consulted, ranging from leading financial services, and health care to insurance providers. All these organizations have faced a hard time in defining a strategy and solution to their “Data Archiving and Purging” initiatives.
If I am to list the top reasons, as per my observations, on why these organizations struggle with these initiatives, it would be as follows:
♦ Lack of better insight into the data collected and managed within the BPM solutions.
♦ Concurrence between different Stake holders (Legal, Business Operation, Data Management and IT) on the requirement of data retention and accessibility.
This leads to cost over-runs of archiving and purging initiatives as well as slipping schedules.
Slippage in archiving projects, as always, has a ripple effect in BPM applications, and if I were to list a few of them, they would be as follows:
♦ Major Impediment to regular data backup and maintenance.
♦ Performance of solutions degrade.
♦ Increase in cost of infrastructure to maintain accumulated data.
♦ Non-compliance of regulations.
♦ Delays in upgrades and migration initiatives related to software and hardware infrastructure.
In this series of blog, I would try to elaborate the approach and solutions related to Arching and Purging data within BPM.
To start with, let us first get to some of the basics of Data Arching, so What is Data Archiving?
“Data archiving is the process of moving data that is no longer actively used to a separate data storage device for long-term retention. Data archives consist of older data that is still important and necessary for future reference, as well as data that must be retained for regulatory compliance.”
— Courtesy World Wide Web.
First, the word of caution would be to never jump into the solution before understanding the key characteristics of the BPM solution that you are dealing with. These are the few high level steps that I would recommend.
Step 1: Understand the data and business process managed by the solution.
Step 2: Develop an insight into the Regulations and Compliance aspects of Data Retention
Step 3: Untangle the knots around the requirement of Data retention and Accessibility
Step 4: Understand the infrastructure and the existing Archiving solution
Step 5: Define the need for Archiving
Step 6: Negotiate the requirement from stake holders
Step 7: Draft a general approach and solution to Archiving
Step 8: Specialize the solution to each BPM applications.
We would explore each of these steps in detail in this series, but before that, I would request everyone to remember that the common design mantra for BPM is “Do not use BPM application as System Of Records”.