Background on FI’s approach to IP&G scoring
At FaithInvest, our mission is to guide faith-based asset owners in aligning their investments with their beliefs, teachings, and values.
A critical component of this process is the assessment and scoring of Investment Policies and Guidelines (IP&Gs, also referred to as Investment Policy Statements). This involves an analysis and evaluation against our comprehensive faith-consistent investing (FCI) framework, which examines various factors including faith-alignment, impact, and screening criteria.
Our Good Intentions study highlights our FCI study methodology and provides summary statistics, representing our evaluation of over 100 investment policies and guidelines to date.
The role of AI in evaluating and writing IP&Gs
We recognise the transformative potential of generative artificial intelligence in the realm of faith-consistent investing. Last year, our FCI postings delved into how AI could assist in writing IP&Gs – only mildly assisting with the drafting of the language used within our FBAOs investment policies. More recently, last week’s Navigating AI: What does Pope Francis have to say? explored the topic of AI and faith further.
AI tools can analyse vast datasets and identify key patterns, but this does not ensure that investment policies align closely with faith-based values. As Michael Lustig pointed out, 'What ChatGPT lacks is human judgment and creativity. The output that was produced is essentially a list, with no context… In short, there is no soul'.
In other words, AI can assist in the drafting of ideas and concepts that relate to faith-values; but it falls short of providing well thought out, contextualised faith-values and principles which can be referenced and applied in an IP&G.
AI for evaluating FCI policies
In my recent work, I have been experimenting with AI, specifically GPT-4, to assess the IP&Gs of our faith-based asset owners. By feeding data from our FCI study and publicly available IP&Gs into the AI, I aimed to teach it our FCI scoring methodology and criteria. This involved a learning process where I inputted 40-50 prompts explaining our FCI scoring process, including what Y (Yes), N (No), and U (Uncertain) responses meant for each of the FCI ‘indicators’ included in our analytical framework. I was able to confirm that the AI understood the FCI methodology and scoring process.
Despite these efforts, while the AI managed to score one IP&G with 75% accuracy (highest accuracy tested), it struggled with all others tested. In one example, it inaccurately identified a ‘Yes’ response for a particular indicator, even though that language was not present in the source material. This highlights AI’s current limitations in understanding the nuanced criteria specific to faith-based investing.
Inviting FBAOs to engage with FaithInvest
At FaithInvest, we understand the importance of aligning IP&Gs with faith-based values, and we would be delighted to assist! We invite faith-based asset owners to send us their IP&Gs for a free scoring and assessment. Our comprehensive FCI assessment service not only evaluates your IP&G against our comprehensive framework, but we also provide detailed guidance on how to enhance and improve it – to best reflect your beliefs, teachings, and values.
PS: For more about AI’s limitations in assessing faith values, read on!
The challenges faced by AI in accurately evaluating IP&Gs can be attributed to several factors which are inherent to how large language models (LLMs) operate. LLMs, like GPT-4, are essentially advanced pattern recognition systems that leverage vast datasets to predict and generate text.
While they excel at processing and generating language, such as assisting in writing an IP&G, they lack the genuine understanding and contextual awareness necessary for tasks demanding deep comprehension of specific faith traditions. This becomes evident in tasks requiring such an understanding, such as FCI.
AI can mimic language and structure but falls short in applying human judgment and creativity needed for nuanced and contextual evaluations. Additionally, the inherent bias in the training data and the absence of real-world experience further limit the AI's ability to accurately assess complex, faith-based criteria.