How to Significantly Improving Acquisition Modelling Performance: The Ability to Create Individual-Level Variables

Predictive analytics practitioners will universally agree on data limitations as being the most significant challenge in building acquisition models.  Traditional solutions have utilized aggregated postal area type data either from Statistics Canada or a range of other data service providers which in addition will also provide cluster type variables. In using external data sources such as aggregate postal area demographics only, we do indeed obtain solutions that yield the ability to rank order response rate. As a result, the use of external data sources has always been the critical element in building these type of tools. From the success of using this data to better target prospects in becoming customers, many of the data service providers in this area will also offer predictive analytics solutions as a complementary service to the actual source data. Companies that desire to obtain more targeted acquisition results look to these providers as being their partner in delivering these results.

All practitioners will profess to the significant improvement in predictive analytics results when using individual-level information. One just has to look at customer models such as cross-sell,upsell, and attrition models in order to see the significant improvement in model performance. Because of the greater success here, predictive analytics efforts in many cases have been  more  focused on existing customers with neglible effort on acquisition targeting. But what if there were opportunities within the acquisition data itself that allowed results to improve significantly.

The key in creating these opportunities is to ultimately create that all-important individual-level information from name and address and thereby create new individual-level variables. Of course, the question is how does one do this ? Working with one of our business partners, we were able to create algorithms based on name and address that resulted in gender and age type variables. Furthermore, information was created on the source of the list that yielded the prospect name. As well, we also overlaid the traditional external postal area demographic information to our prospect list and then began our model building process. In our final model, nine variables were identified as the key predictors of acquisition response. Interestingly enough, the external postal area type variables  were the two weakest variables in the model. The model in terms of performance yielded a performance lift in acquisition response of 5 to 1 between the top 5% scored names and the bottom 5% scored names. Using just postal area demographic variables yields only a performance lift in acquisition response of 2 to 1 between the top 5% and the bottom 5%. The use of individual-level information in this case clearly provided that additional response lift advantage.

Of course, more work in employing more  practical examples need to occur before overarching conclusions can be made regarding this approach. But nevertheless, this initial work in this isolated case does point to the need that this approach should be clearly explored in other acquisition type programs. Our belief in identifying significant improvements to predictive analytics solutions has always been less about the mathematical or advanced statistical approach and more about the actual data itself. Our work in this area with this client certainly amplifies this belief and it is our hope that more organizations at least explore this approach of converting name and address information to individual-level information  as a viable option in improving their targetted acquisition efforts.

 

The Data Discovery: Investing in Customer Insight

            The Data Discovery: Investing in Customer Insight

In many of our engagements with new clients, the old Donald Rumsfeld phrase of ‘We don’t know what we don’t know” is very applicable as these organizations commence their journey into database analytics.  In these situations, there is no clear definable objective or goal when undertaking these projects. In fact, these companies look for outside consultation in the creation of a roadmap which represents both strategy and tactics on what database analytics projects they should undertake. These type  of exercises don’t yield an immediate return on investment which can be a barrier to many organizations who fail to appreciate the long term benefits that a data discovery can yield. It is often very difficult to convince these organizations of the longer term benefits of database analytics as they are focused on achieving  short-term gains to resolve an immediate business need. In many ways, this is the true challenge of a data discovery as we seek to strike a balance between the longer term analytics needs versus establishing an immediate ROI.

In undertaking these type of projects, the only common feature is the open-ended nature of the assignment  as an end solution to solving a specific business problem is not necessarily our goal. Instead exploration and discovery is the intent of the project with the goal being to build an analytical roadmap. Yet, even open-ended projects require some process in order to provide guidelines and steps which are necessary for its success. Typically, this process involves four steps which are:

·        Preparation

·        Data Audit

·        Preliminary Analysis

·        Recommendations

Preparation

The first stage of this project represents the portion of the project where the analytics practitioners attempts to increase their  knowledge of the client’s current business and results.  In data mining and analytics, all experts  agree that analytics projects require both domain knowledge and data mining expertise in order to really optimize a given solution. Domain knowledge is specific knowledge which pertains to that business. It represents knowledge which is both unique for the industry sector(finance,retail,etc.) of the client business but also knowledge which is unique for the mechanics of how that client business runs. The preparation stage of the project allows the practitioner to increase their domain knowledge of this business.  Of course, the domain knowledge of the practitioner will never be as exhaustive as the client but the objective here is to obtain an adequate level of this knowledge in order to conduct an effective discovery exercise.

The initial tasks here are to conduct extensive interviews with key business stakeholders from marketing, I/T, analytics(if there is an area), finance, and the executive depending on availability. During these meetings, key business issues and challenges are identified  as well as an understanding of what data is available. Business reports/analyses or any other documents that provide results and meaningful information about their business are shared with the practitioner. At the end of this stage, a data extract is then requested which consists of all the files and fields that will be required for the remainder of this project.

Data Audit

Data audits have been discussed in the past and are a core prequisite in any data discovery exercise. At this stage of the process, the practitioner attempts  to become “intimate” with the data which describes a much stronger relationship with data than the standard phrase of “ data knowledge”.

Once the data extract is received by the practitioner , the data is then loaded into their system whereby  standardized reports are produced that essentially provide the following results:

·        Data completeness or coverage  as indicated by the number of missing values in a variable

·        How do  values or outcomes distribute within a  given variable

·        Data inconsistencies and data gaps. Do values change overtime and are there groups of records where certain data anamolies exist  

From these results, the quality of the data can be assessed in terms of determining what information to use in a future analytics exercise. The results also yield files would be linked together since one objective in this phase is to create an analytical file where we have one record per one customer. With these links determined on how the analytical file would be created, the variable creation exercise can commence. This represents the most labor-intensive portion and arguably most important component of the  entire discovery process as it is here that meaningful variables are created which  can be used in any future analytics exercise.

Besides the exhaustive reports from the data audit, a summary level report is produced which indicates the major findings from the data audit. This would indicate what are the current gaps within the data environment and how it might be improved. Some of these gaps could to some extent be filled by data overlays. Good examples of this are Stats Can data for business to consumer type analytics and perhaps Dun and Bradstreet or Info-Canada data for business to business type analytics. A good example of how data overlays might fill a gap is if income was a key component in any analysis but simply unavailable within the current data environment. Its use here at the postal area level as opposed to being used at the individual level would be  a secondary option in attempting to derive insights based on income. Once this data audit exercise is completed, some preliminary analysis is then conducted which essentially commences our journey into analytics.

Preliminary Analysis

Note how we use the word preliminary here as our objective is to acquire a basic level knowledge or understanding of their current business using our data analytics expertise complemented by our domain knowledge. Obviously, as time goes on and the analytics discipline becomes more entrenched, the level and sophistication of knowledge will increase. 

Our first exercise is to understand at a very basic level how customers are different. A value segmentation or RFM segmentation is a tool we use to stratify customers into deciles with  decile 1 representing the highest performing and  decile 10 representing the lowest performing  customers. From this stratification, we can then arrive value-based or RFM-based segments. Under each scheme, though, we can determine how relevant the 80/20 rule is(i.e. do 20% of my customers account for 80% of my business).

This analysis also helps to point out the migration and retention opportunities. For example, a sensitivity analysis using results from this segmentation analysis and a series of different migration(conversion) rates  would determine the $ opportunity in migrating customers from a lower performing segment to a higher performing segment.  Retention analysis would use somewhat the same approach by looking at  a high performing segment and then determine the $ lost when looking at different potential defection rates. Of course, any sensitivity analysis which looks at a range of  migration rates or defection rates should be reasonable. The reasonableness of these numbers, though, would have been confirmed through  results and comments from key stakeholders that were gleamed from the upfront preparation stage.   

Once these segments are determined, profiling of each of these segments helps to better  understand the key customer characteristics that comprise a given segment. Profiles can be used in two ways. Strategically, the information can be used to develop communication and channel strategies which are more engaging to the mindset of that particular group. For instance, a particular profile of a high value customer may comprise the following characteristics:

·        Older

·         wealthier

·        Tend to engage more in  online media

A heavier online strategy speaking to the concerns of wealthier retirees or people nearing retirement would certainly be very appropriate here.

The second way is tactically where the information is used to create lists of customers that most resemble the profiled segment. Here, we target those low to medium value customers that tend to be older, wealthier, and more engaged online since these customers exhibit the characteristics of a high value customer.

Cohort analysis represents another type of unique analysis as its purpose is to explore  new customer behaviour  over a number of years. Typically, this kind of analysis commences with customers that were new five years ago and then tracks their spending behaviour behaviour in subsequent years. The same analysis is then conducted in subsequent years. Obviously, as we move  forward in time, we have less tracking history with new customers. Through this kind of analysis, we can begin to discern retention,migration,and upsell patterns amongst these new customers. At the same time, we may also find that there are unique behavioural patterns that are associated with a certain cohort group.

Depending on the nature of the industry, basket analysis is another form of analysis to better understand the “event” behaviour of the customer’s purchase. The analysis can reveal that the mix of products purchased can vary between customers. For example, this  product mix will in most cases vary depending on whether they are a high medium or low value customer. Migration opportunities can use the insights generated from this analysis by promoting products that are first relevant for the lower value segment and secondly that will migrate them to the higher value segments.

New customers will also exhibit different  product mix patterns  when compared to older customers. As indicated above, the same migration type of migration analysis is conducted here by identifying those relevant products which should be promoted  so that new customers are on a path towards becoming higher value customers.

Recommendations

With the completion of the preliminary analysis, the actual required work for this discovery exercise is now finished. It is now time to consolidate all this work into a comprehensive document . This document contains all the actual details concerning results which are used to support the many findings and insights generated from this exercise. But  despite the comprehensiveness of the report, it is of no use if it does not clearly lay out actionable next steps.  These actionable next steps are focused in two areas. The first area consists of the data strategy and what the organization needs to do to improve its information environment in order to more fully optimize the analytics discipline. The data strategy needs to be proactive in addressing not only the existing  analytical needs but also any potential analytics needs due to the changing nature of analytics.   

The second area focuses on what specific analytics activities should be undertaken in the first year and in what priority so that the company can effectively commence its analytics journey. The analytics strategy should be short-term in nature so as to provide an initial benchmark from which to commence this discipline. Keeping a  shorter term perspective on analytics just simply recognizes the underlying fact that things can change very quickly within organizations. Obviously, this short-term focus will identify activities that will yield the quickest wins in terms of ROI but also activities that will generate much learning.  It is this second component of learning which ensures that the longer term ROI is not being overlooked.   

By having both a data strategy that optimizes the use of analytics complemented with a set of analytics activities that begin the organization’s analytics journey, the organization has a roadmap or path in which to begin this journey.  With this kind of mindset, investing in a data discovery exercise is an easy decision as long as this process is undertaken with the due diligence and discipline as outlined in this article.

Using Analytics to create Rating Territories

One of the many areas of property and casualty risk analysis is the effective creation of rating territories based on some measure of risk.  In Canada and the U.S., geographical areas are grouped into territories based on claim risk. For example, postal codes in Canada and the comparable equivalent of zip codes in the  U.S. are grouped into rating territories.

As the use of analytics continues to adopt the  more scientific or mathematical techniques, insurance organizations would be remiss in not attempting to deploy these disciplines. However, before even adopting these disciplines, the typical rigors in developing an analytics solutions still need to be conducted . This means that the information or analytical file needs to contain the right data before even using any of the more advanced techniques.  In properly assessing risk, it is self-evident that any rating assessment should reflect the most recent information. Under this scenario, the determination of a proper post period is critical in the development of this solution. For instance, do we have enough data to look at the last 12 months or do we need to broaden the period to 24 or 36 months. For smaller organizations, this can be a significant factor as analysts need to have enough claims in order to build an effective solution. More claims for analysis in theory can translate to more robust models.

 Another factor in this determination may be understanding the cyclicality of claims within a given business. AB claims can often take more than 12 months to determine their exact payout. If loss cost is the key determinant rather than claim frequency, one would want to look at a longer timeframe beyond 12 months  in creating rating territories particularly if AB is a significant driver of overall loss cost. The compromise here is that we are sacrificing recency of information versus information that is more reflective of the specific business conditions of the company.

Besides this determination of the proper post period in analyzing claims, one needs to consider information that is considered extraneous to the solution. Often this  so-called ‘extraneous’ information refers to claims that are classified as catastrophic.  The inclusion of  auto claims that are due to a hailstorm as part of a solution will introduce information that is not really indicative of any behavioural tendencies. Instead these tendencies are solely due to ’Acts of God’. Yet, it is behavioural tendencies that we want to better understand and more importantly to action in future business initiatives. Grouping of geographic areas around behavioural  tendencies yields consistency when projecting this learning going forward. ‘Acts of God represent noise or variation in the data that should not be considered in any analysis. Accordingly, these kind of observations should be excluded from any insurance risk analysis whether it be the development of pricing models or the determination of new rating territories. 

Once this analytical  file is created, the ‘analysis’ portion of this exercise can commence. The information in such a file needs to contain only two fields of information with one field containing loss cost or loss frequency  while the other field contains the specific geographic area.

One creative approach is to use a technique that is traditionally used to develop models. CHAID which stands for Chi-Square Automatic Interaction Detection represents a non parametric approach to building models. Essentially, the solution is a decision tree whereby the nodes or branches represent segments comprised of business rules. Comparing the traditional parametric modeling such as equations with variables and weights assigned to each variable(parametric approach), CHAID’s solution of segments and business rules(non parametric) represent the model solution. Nodes or segments  can be ranked sequentially from the highest loss rate node to the lowest  loss rate node and policies can selected in this manner. In most scenarios,though, there will be more than 4 nodes which allows for more granularity regarding the selection of policies. 

However, one real significant advantage of CHAID is its data exploration capability. Through CHAID, certain character type variables can be grouped into meaningful segments based on some objective function. One obvious application is the grouping of character type data such as  geographic areas into meaningful segments or territories. Here the node output represents groups or segments as determined by the CHAID routine. Loss rate ,though, is the key metrics which drives this statistical output.

Certainly, the use of CHAID as a tool for territory assignment is not to imply that what was done in the past was incorrect. Due diligence has always been exercised by combining both the analysis of historical loss rate data as well as the domain knowledge of the actuary. But rather than discard what has been used in the past, we simply have another option which is more mathematically based on how geographic areas are assigned into territories.