Do you have a set of complicated, handcrafted Excel workbooks that perform parallel calculations to test vendor or proprietary software? As you may be aware, these testing tools can take on a life of their own, especially in situations with numerous product variations, options, or switches. Sometimes, when converting from one platform to another, detailed calculations can be directly compared between the two platforms. But often, custom tools must be created to evaluate certain calculations, usually related to contract mechanics.
One way to significantly reduce the cost, in both dollars and time, of this process is to use AI tools to develop the parallel models. We will explore a process to do that in this article.
First, however, we need to address the concern that is popping into your mind right now. Either AI overconfidence and/or hallucinations have burned you, you have heard secondhand about this, or you are just skeptical and unsure in general.
Newsletter continues after job posts…
👔 New Actuarial Job Opportunities For The Week
We post 50 new handpicked jobs every week that match your expertise.
Here are a few of the new jobs this week:
Aon - USA - Forensic Accountant (Claims Preparation)
Salary Range: $200k - $250kArgo Group - USA - Lead Capital Actuary
Salary Range: $151k - $198kState Farm - USA P&C Actuarial Analyst
Salary Range: $82k-$82kCanada Life - Canada - Investment Analyst
Salary Range: $57k - $86kCanada Life - Canada - Assistant Manager Investment Financial Reporting
Salary Range: $60k - $100k
Interested in advertising with us? Visit our sponsor page
The key fact is that AI is producing human-readable code, not results. For this purpose, if the results match and you have been careful to ensure independence, you are done, and no hallucinations are present. The exception to that statement is when the source documents are incorrect; both models are incorrect, since the validation exercise verifies conformance with the documents rather than their correctness. If the results are off, both models need investigation, which is what parallel testing has always needed anyway. I can’t overemphasize the importance of independence; it is created during the construction phase and, once lost, cannot be regained.
Let’s walk through a typical approach:
Step 1 - Get ready
Product Specifications: Identify the product mechanics from the source documents, for example, a single premium FIA with annual point-to-point crediting, a cap and floor, and a declining surrender charge schedule.
Source documents used as AI inputs: the policy form and the actuarial memo, with any sensitive info redacted. No primary model code, logic, or descriptions of such are allowed anywhere near the AI to retain independence.
Construct a deterministic index path used by both models exclusively for this test. It may be obvious, but you want to keep inputs identical between the two models. Otherwise, you are adding differences that have nothing to do with product mechanics.
Choose a single cell, say age 60, and a projection length of 15 years.
Decide whether a per-policy illustration or a decremented projection is needed.
Make sure you are following your organization’s (or your clients’) confidentiality and retention guidelines.
Keeping vendor code and logic out of the AI is important not only for independence but also for compliance with vendor license agreements.
Where the model runs is a procurement question rather than a methodology one. Public AI may make you (or your bosses or clients) nervous. If that is the case, you would need an enterprise tier with zero retention, the client's own tenant, or a self-hosted model behind their firewall.
Step 2 - Build a Model with AI
Add source documents to AI. Make sure to purposefully omit any items related to the current model to ensure independence.
Prompt AI to create a new structured specification document, where each item lists its source document or absence from the provided documents.
Ambiguities, such as items missing from the provided documents, are to be reviewed by the product actuary (not the modeling team) and rectified prior to the AI model creation.
Assumptions (as opposed to product specs) are read from external documents if needed. As was done with the specs, have AI produce a list of assumptions, with a source (or lack thereof) for each item.
Tell AI to go ahead and create the model. Beware human leakage at this point. This means that, since the human prompt builder may also be familiar with the primary model, he should not provide “assistance” to the AI. As stated in #3 above, the product actuary should be consulted on any ambiguities. The theme of independence is key to making this approach work.
Run AI-created model against degenerate cases with hand-derived answers as a preliminary check.
Please take note of #2 above. We are asking AI to find any items that may have been left out, which is a key difference from the traditional approach. This adds additional value to using AI in the process. For instance, the memo may never state whether a contract terminating during the policy year earns that year's index credit. This sounds like a minor item but could have major modeling implications if missed.
Step 3 - Run a reconciliation
Compare models item by item at every period, with a stated tolerance. Items may be interest credited, index components for a policy illustration or include things like surrenders for projections.
Construct a table with item values by row. Make note of what ties and what is materially off.
Step 4 - Diagnose
Narrow down what is different by first eliminating what ties out.
Look for patterns in the findings - like constant ratios of differences, or notable patterns of differences by duration. These will give you clues to what is occurring.
Example: If death and surrender benefits are off but policy values tie, you have a decrement or projection issue to investigate. You will need to use your judgment and experience to resolve this. Of course, AI will be happy to give you an opinion as well if you ask. The idea here is to check off the areas that tie and narrow down the differences.
Step 5 - Investigate
After the diagnostic step, we check the sources of model differences against the documents. This is done separately for the primary and parallel models.
Freeze the parallel model at this point for independence. This will prevent any primary model-related information from leaking into the AI-produced parallel model.
Fix both models using the documents, keeping each isolated from the other. At this point, you are likely working with Python code created by AI in the parallel model. You can have AI analyze it but give no hints from the other model
Now you will have an independent second implementation, which has always been a solid validation technique. It will be accomplished with greater speed and lower cost than before. Please note that the parallel model is itself a model and carries its own governance obligation under ASOP 56. That means it needs its own documentation and its own validation record.
The bottom line is that the tools have changed and costs have been reduced, but not the method. It’s hard to nail down specific time savings, since there is a learning-curve overhead and dependence on product complexity and document completeness. The method above was implemented in a tool-agnostic way. The requirements for the AI tool are basic: it needs to accept document uploads, generate readable code, and have well-documented retention terms.
John Hegstrom, FSA, MAAA, is an actuarial consultant with 40 years of experience in life insurance and annuities across valuation, pricing, and model architecture, primarily using Prophet and Axis. He runs Actmod Solutions, LLC, which focuses on actuarial modeling for life and annuity insurers, with an emphasis on AI-augmented delivery.
Connect With John
Send a Message | Business Website
Looking for clarity on consulting, income, or next steps?
Read my new book, The Independent Actuary - a practical guide for actuaries looking to build more income, leverage, and career optionality beyond the traditional path.
Last week we covered The Actuary as a Capital Allocator.
👉 If you missed the last week’s issue, you can find it here.
💼 Sponsor Us
Get your business or product in front of thousands of engaged actuarial professional every week.
💥 AI Prompt Of The Week
Pricing Scenario Brainstorm
Stress-tests your thinking before you build a pricing model. Identifies the assumptions that drive profitability and the scenarios that bracket the downside.
The Prompt:
I’m pricing [product]. Before I build the model, help me stress my thinking: what are the five assumptions most likely to drive profitability, which one is most uncertain, and what scenarios should I test to bracket the downside?
