Key Steps to Migrating Policies Compliantly
With a number of firms upgrading their approach to monitoring and supervision, or simply moving to a new vendor because their legacy technology is at end of life (e.g. Orchestria/DataMinder), we explore the process needed to achieve a successful migration.
There is no one better equipped to guide an enterprise through this tricky transformation than Technically Creative’s Danielle and Christopher Amatulli. They have more than 14 years of experience with the Orchestria/Data-Minder/CA Data Protection product and are one of the industry’s leading experts in policy migration and development. To say they can do this stuff with their eyes closed is underplaying it!
Identify policy stakeholders
Each firm has different stakeholders when it comes to policy, but the two main categories that are common are those responsible for policy definitions, and those responsible for sign-off on acceptance criteria. In some firms, IT, compliance, control room, legal and other groups may also have a stake in policy. It’s important to accurately define the sign-offs and process for testing and releasing policies.
The migration process is easily simplified if you have an existing process in place and those policies are properly defined. In some cases, the destination vendor may be able to pick up that documentation and greatly improve the migration success. But beware – the main policy stakeholder is the one person who is on the hook here.
Document the policy details – need, purpose and correlation to the regs
Detail who the policy is being applied to (global; certain groups; one office). Is anyone excluded from the policy, and if so, why? Identifying the people that the policy applies to is critical – anyone missed out is a huge potential liability. It’s a common fault at firms; surveillance gaps appear due to missing or even incorrectly labeled identities. What content needs to be captured? What is the overall objective of this policy? Are you looking for insider trading? Or people discussing a reward related to their work or a client, or even a new rule on incentives? Write down what you are targeting.
Where are we looking for this content? Is it in the email subject line, the body of the mail, or an attachment, in web activity, in a file on a shared drive, or sent to a printer?
In what circumstances does this policy apply? When sent externally, or when one individual sends to another individual in a certain department, only internally, or when a person in the research team is included? Does an attachment need to be present? Perhaps it is when the mail is sent out of normal business hours.
Why is this policy being created? Is there an internal risk? Is there a regulatory concern this policy is associated with? Did something happen previously at the firm like this? All of these questions and the documentation of your policy are essential for making sure it is well defined and matches the firm’s individual risk assessment.
Fill the gaps – is there adequate policy coverage for the supervision requirement?
The introduction of a new solution is the perfect opportunity to review current coverage and identify any gaps. Risks are different for each financial firm, including the way that the business is operated. Previous public enforcement, as well as issues distinct to each firm, are a great guide to start analyzing the coverage.
Specifically examine the language covered in the policies, whether basic or more advanced. Is “parking and wash trades” a concern at the firm? Or “trading ahead of research”, or policies for deceptive or inappropriate language? The personnel on the frontline are best placed to determine the core issues to target, and firms should create policy to ensure wide coverage, not forgetting narrower policy to cover special situations.
Understand the differences in terminology and functionality
Orchestria’s language is often referred to as a lexicon, but it is far more than that. In technical terms, it is essentially an NLP language that is typically associated with machine learning. It requires a manual approach and some quite complex development effort to build and maintain it. Some users of Orchestria neglected their policy effort or used simplified lexicons in place of utilizing this language. This can help in a migration, but it doesn’t help in ensuring supervision is complete, covering the topics of concern.
With the latest technologies, the break away from lexicon dependence seems to be the direction most are heading; this has been difficult for some compliance folks to accept due to obvious questions around how the system works. It need not be a deal breaker. Transparency around the lexicon list and wrappers, including the context to show what is being targeted, can be just what the regulator or auditor needs to see.
As an example in Orchestria, “policy” is the entire rule set for supervision. Policy incorporates multiple “policies”, which contain the criteria targeted. In another vendor, a policy might be a supervision model which would incorporate identities of who must be reviewed. In another, a policy might be a single rule, more akin to a lexicon/word list. At a more advanced vendor, the policy is usually an advanced combination of multiple AI/lexicon/graph and NLP libraries.
The policy lifecycle
The policy lifecycle, regardless of product, for both development and refinement, is fluid and requires constant effort to not only eliminate junk or false positives, but also incorporate change in regulation and market practice. Language needs amendment or perhaps new policies are needed to increase coverage.
The policy development process is notoriously tough to get right. Some firms don’t have such a thing. Start with analysis and look at what exists and compare it with what should exist.
As an example, there may be a policy for customer complaints, but it doesn’t include terms like “you messed up” or one for monitoring gifts and entertainment without the word “gift” in the policy. Several firms were cited for this in recent years. On its own the word gets a lot of hits, but the trick with any good supervision solution is reducing the hits by excluding the cases where it has no relevant context.
During the analysis phase, the stakeholder must be involved. This includes anyone that determines what a policy needs to cover or the concerns of the firm in its communications.
In the design phase, the key objective is to identify what needs to be found, and then to document it correctly so that the policy can be built.
Policy construction is the hardest element. Resources are needed to set up the system, establish data sets, actually construct the policy, and then run data through the system to test and validate the results before considering the refinement cycle.
In the refinement phase, conduct a review of the results of the constructed policy and be ready to potentially run through multiple cycles of ingestion, reviewing the captured content. The release phase follows this, which is arguably the highest stress point when all of these changes get put into production. It is always essential to have test users and test events for the policy, also incorporating any changes to run through the system so proper capture can be confirmed.
The last phase is validation and reporting. It is best to be proactive and run analysis and trending across all of the changes, accounting for example for capture rates, frequent violators and frequent language results in the hits. A week of data might kick up huge rises in hits because of a simple phrase in common documents which gets captured. Quick refinement will increase the popularity of the compliance tech team among the reviewers!
In summary, the policy lifecycle is probably the most critical piece to the health of your supervision system. Many firms experience issues where a policy fails after changes are made that render it non-functional (often when key words or phrases get removed). Having an effective policy lifecycle, where changes are evidenced and validated with no loss of true positives, is critical to staying out of the crosshairs of the regulators.
Policy development strategies – ensuring proper coverage in the new system
Convert: One approach is to extract the lexicon and phrases from the policy and try to have the destination system interpret these. This will involve extensive testing and validation.
Migrate: Another approach is to manually recreate existing policies in the new system, based on existing SMEs’ knowledge. Migration should result in a better outcome due to the volume of testing required on the policies. In the process, issues in the current policies can be refined to improve them in the destination system.
Build: An alternative approach involves building new policies but doesn’t necessarily mean starting from scratch. Some vendors have “out of the box” policies, which provide the starting point from which to add or modify them to make them really effective for the firm.
Validate and test
It’s critical to confirm policy effectiveness in any new system by testing the results of policy conversions from the old system. It is important to get on top of false positives as they sap the morale and productivity of the reviewers, and this increases the risk of potential oversight or a tendency for bulk reviews/missed flags. Some typical glitches with a new system might be where policy scoring wasn’t taken into account with the positive and negative indicators, or several of the phrases were interpreted as stand-alone words.
Don’t underestimate the risk here after all the hard work that has been done leading up to this part of the project. The migration, creation and maintenance of the policies are paramount. The differences in the way language is interpreted or the new functionality utilized can put in place some fatal oversights. The key is to test, test, and test again. The regulators will want evidence of validation when moving to the new system, and this is an essential part of your due diligence.
Plan for maintenance and refinement
Policy must be adapted and maintained; regulators don’t accept out of the box policy. This starts with the sign-off and acceptance process, ensuring any change has names associated for approval. Any ticketing system needs to be archived. A refinement report will detail what line was changed/added/removed and why. Next comes a comparison document, which is a literal release document showing the “before and after” of the policy, which must align with the refinement report. Last but not least is evidencing the policy change. This involves maintaining the output of items that triggered the policy before as opposed to after. And then you’re done! Simple, right? •
Technically Creative is an IT services provider specializing in electronic communication surveillance, data loss prevention, and technology solutions for compliance. They have experts specializing in data conversion, data integration/ solution integration, identity management and incident work flows as well as enhancing existing solutions to meet regulatory obligations.
Danielle Amatulli is the CEO and Director of Business Operations at Technically Creative and orchestrates successful partnerships and projects through information, organization, and innovation. Christopher Amatulli is the CTO and Director of Services and Architecture at the same consultancy and specializes in advanced technical implementations, systems architecture, and integrations. Technically Creative is a certified CA Technologies developer and support partner. Their team of engineers has more than 14 years of hands-on expert knowledge of Orchestria / DataMinder / CA Data-Protection product.
For more information on policy migration or other services, you can visit Technically Creative’s website at technicallycreative.com, or by emailing the team directly at [email protected].
