Analytics

Run DMCMC

/ rymerben / Leave a comment

Run DMCMC

Turning the geek factor up to 11 for a moment, there are some interesting possibilities for mathematical techniques used in technologies like predictive text to be used to assess fundraising interventions.  Ever since an influential 1948 paper by Claude Shannon – “the Father of the Information Age” – so-called ‘Markov Chain’ models (a variant of which is  ‘Markov Chain Monte Carlo’, or MCMC) have been “widely used in speech recognition, handwriting recognition, information retrieval, data compression, and spam filtering”, as well as ‘Natural Language Processing’/word prediction, by assigning probabilities to ‘state transitions’, ie the probability of one letter or word following another.  Using such chains to predict which fundraising interventions are most likely to lead to a gift would be a huge boon for the industry, leading (in theory at least) to far more efficient donor journeys and more granular understandings of business process value.  So, who wants to Run DMCMC?

Ratio’s(ination)

Imagine an ant crawling along a beach, left and right, forward and back, up and down, as it navigates home. It’s chosen path looks something like this:

Ant walk pic

The route is complex, but the complexity is a product of the environment, not the ant, whose decision-making power is minimal.  The example is abstract but relevant for people, too: “human beings, viewed as behaving systems, are quite simple. The apparent complexity of our behavior over time is largely a reflection of the complexity of the environment in which we find ourselves.”

The question of how to navigate complex environments using limited decisionmaking capacity and incomplete information is at least as relevant for organisations as it is for animals.  As a fascinating recent post [login required] to the Prospect-DMM email forum suggests, the answer may lie partly in the use of ratios, which offer an elegant, contextualised ways to cut through bewildering amounts of information.  Simple, powerful metrics to use in fundraising could include:

  • Last five years giving/lifetime total
  • Responses/contacts
  • Number of appeal/number of gifts
  • Cost of appeals/lifetime donations

One obstacle is not being able to integrate or even extract information from our database systems to begin with.  Recent news that insurance giant Aviva has made great strides in integrating database systems to the great advantage of their business raised a thought which is highly relevant for many charities: are we prisoners or masters of our IT/database systems?  And, when techniques like database screening may be restricted or even off-limits in future, can we afford not to try to mine other data for insights?

Weapons of Math Destruction

If, as the ICO believes, the British public would experience “substantial distress” in learning their data had been processed in a wealth screening, the public will surely be distraught should they ever read Cathy O’Neils 2016 book Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy.  The many ways in which mathematical models and algorithms – the so-called ‘WMDs’ – are used to make crucial decisions relating to the public realm and, increasingly, private lives are as worrying and widespread as they are opaque and unaccountable.  Across vital issues like criminal justice (where court decisions increasingly use automated quantitative modelling and scoring), access to credit, finance and education (where credit scoring and rating of teachers increasingly rely on WMDs), jobs and employment (where a missed payment could mean being overlooked for a job interview) and even the feelings and emotions we experience (thanks again, Facebook), WMD’s are in wide and growing use.  This largely unseen trend is worrying as WMD’s inevitably contain errors and anomalies which, if not caught, can have significant effects for those affected by their scores or results.  Even worse, WMDs can have pernicious effects when they run perfectly – many contain implicit value judgements which end up disadvantaging poorer groups, or, in the case of aggressive advertising, are designed to target these very people.  Yet all too often WMDs’ methods and results go unchallenged.

O’Neils Mathbabe blog is an engaging mix of political commentary, engaging geekery and knitted hats – well worth a read.  And both are valuable and timely in helping us to understand – and hopefully better manage – our algorithmic overlords.

Where is the Money (Going to Be)?

In No Country for Old Men, menacing assassin Anton Chigurh (Javier Bardem) shuns Woody Harrelson’s frightened offer of help to find a satchel loaded with millions of dollars.  “I can find it from the riverbank”, a terrified Harrelson pleads at gunpoint, “I know where it is”.  “I know something better”, counters the icy Chigurh, “I know where it’s going to be”.

Chigurh Hotel Scene pic

As fundraising researchers, we spend a lot of time focusing on where the money is.  But do we spend enough thinking about where it is going to be?  The scene is a reminder that to prospect by relying on companies or sectors enjoying current success (as a way to assess employees’ affluence) is to miss a trick.  Do we prospect often enough by trying to predict which sectors will become successful in the future?  It may sound like a fool’s errand, but understanding which sectors and products are on a strong growth path and likely to experience an uptick in growth – wearable tech, virtual reality, voice recognition technologies and peer-to-peer finance come to mind – would be a boon for prospect research.  Intelligence on mergers & acquisitions, IPOs and other comparable ‘liquidity events’ is equally valuable (lookin’ at you, Aramco).  Such horizon-scanning need not be resource-intensive and is par for the course for many investors and businesses – for very good reason.

Calling Bullshit

How to call bullshit in the age of Big Data?  There is now a whole course designed to do just that, and it is the best thing ever (no b*llshit).

callingbullshit

Marianne Pelletier interview: the power of understanding donor engagement

/ rymerben / Leave a comment

 

Error
This video doesn’t exist

Following on from our first interview last year, it was great to speak with Marianne again. She highlights the importance of linking fundraising staff, operations and analysts to instill an evidence-based culture in your team, as well as the importance of being targeted in fundraising activity to raise funds efficiently.  Measuring engagement also comes across clearly as perhaps the major recent trend in understanding donor motivation.  We face a marketplace of donors who often feel bombarded by requests for support so being targeted, in order to build stronger relationships and get the best ROI, is a major challenge for charities.  Marianne gives some practical ways we can achieve this, and raise more in doing so.

The interview is available as an audio file here: https://soundcloud.com/benrymer/marianne-pelletier-fund-raising-voices-interview-part-2

Rare Events

/ rymerben / Leave a comment
Some things don’t happen very often.  Rarity makes them interesting and important, but also cryptic.  Epidemiologists, insurers and bankers, the military, geographers, scholars of international relations and meteorologists all face important challenges in forecasting how likely rare events are to occur over a given period of time.  Rare events matter in fundraising too; major gifts are scarce, but important.  So how do we estimate the chance of big gifts occurring when only relatively few will be given across a supporter base? (Hint: it’s not via a heat map):
white bar padding space
xkcd heatmap
One popular method has been to use logistic regression, a statistical method used to illustrate how likely it is that one variable causes others to change in a given scenario. However, for various reasons, it can sharply underestimate the chance of rare events occurring, making it unreliable in forecasting.  However, a new approach being pioneered by Italian academics Raffaella Calabrese and Silvia Angela Osmetti may have some answers. They propose a new statistical method to account for situations where one comparison group is very small, hopefully bypassing the underestimation issue.  And another area (even more) remote from fundraising or marketing could offer some clues on gaining insight into rare events: namely, the study of civil wars.  A recent paper from Rob Blair, Chris Blattman and Angela Hartman describes how the use of a set of statistical methods including neural networks, random forests and ‘LASSO’, a kind of logistic regression, can help to gain insight into the likelihood of occurrence of civil war.  The paper presents three findings of interest: first, that all the methods used return results better than chance, (some are far better), and therefore have a good degree of predictive power.  Second, the simplest method (LASSO) gives some of the most accurate results (hopeful for the non-statisticians among us), and, third, initial results suggest some novel causes for conflict that previous literature had not highlighted.  These findings are partly based on important earlier work by Gary King and Langche Zeng, whose much-cited 2001 article is a key text in the rare event literature.  This research is a step in the right direction, although obviously does not solve the thorny issue of using data and statistics to predict rare events.
white bar padding spaceFor those who think this discussion should go back to the statistics chat forum it managed to escape from: think again.  Developing innovative methodologies is at the heart of the challenge for fundraising in igniting sectoral growth.  We all know about the 80/20 rule.  But as Peter Wylie recently pointed out, for many not-for-profits, a fraction of a percent of the donor base contributes a huge majority of total giving (more like the 0.1/50 rule).  This is not unique, but is acute.  Wylie speculates that repeatedly asking existing donors for support is the main reason for this concentration, as is a short-term outlook in campaign planning (no doubt wealth polarisation in the wider society also plays a part).  Whatever the cause, we need evidence-led philanthropy to break the 0.1/50 rule, fast.  The FPS and Government austerity could between them shrink British fundraised income significantly.  A tougher regulator has been appointed, big name charities are already forecasting imminent losses and data protection, if not yet nuclear, is developing not necessarily to DM’s advantage.  Smart, evidence-based relationship fundraising with engaged, informed donors can help to mitigate all of these risks.
white bar padding space
panning for gold
white bar padding space
Information is the lifeblood of modern organisations, and philanthropy can suffer due to a lack of robust, granular management information on forecasted donor value.  But this should not be an excuse not to try to understand the predictors of major gifts.  The studies mentioned above work with far more difficult and complex environments than fundraising, and are making headway in their forecasting efforts.  It is still very early days in using probabilities to gauge the effects of specific solicitation or cultivation activities on donations.  While developments in arcane statistics journals will obviously not raise more money in isolation, improving our data-driven prospecting and fundraising probably will.  And when major gifts are less rare, the models will predict them better, making them even less rare, meaning the models will predict them better…

A Lie That Reveals the Truth

/ rymerben / Leave a comment

This is adapted from a presentation I gave last week for Researchers in Fundraising, titled ‘Introducing Data Analytics’.  Enjoy!

Prospect researchers should care about analytics for many reasons.  There has been no rise in charitable donations in the UK or US for decades; charities urgently need to build stronger relationships with supporters; only half of high-value individuals are covered in wealth screenings; it need not be rocket science, and automation is real, and prospect researchers must remain relevant.

The presentation shows what analytical thinking is using the example (originally from Ernst Gombrich’s ‘Introduction to Art’, quoted in John Kay’s book ‘Obliquity’) of Pablo Picasso.  The Picasso museum in Malaga (where he was born) is ordered chronologically, and the level of realistic detail in his work decreased through his career.  He abstracted more to reveal more; he became more expressive by using ‘styilised simplifications’, a term which also describes quantitative models.  To paraphrase him, Picasso ‘lied to reveal the truth’.

Building a team and culture are central to using analytics and being evidence-based.  As Martin Squires, Head of Insight at Boots the Chemist, said at the Insight Special Interest Group conference in 2014, the essential qualities for an analyst are more than anything “curiosity, communication and commons sense”.  This culture can be built from the bottom up or the middle out.  As Clara Avery (Clara Avery interviewee on this site) has said, Macmillan were “probably calling ourselves an evidence-based organisation for two years before we were one.”  Slide eight below shows how Macmillan use evidence at each stage of the innovation process, a process which, as Clara says, took time to grow but is now established.

In terms of methods, analytics for fundraising often identifies a group of supporters, profiling them using behavioural, demographic and/or attitudinal data and looking to the wider population of supporters to try to identify those with a similar profile.  Different organisations will have different supporter profiles; likewise, appeals and products will have different ‘typical’ supporters.  Slide 11 of the deck shows some of what I think are particularly insightful data points, none of which need any great numeracy to work with.  Indeed, none of the methods I’ve quoted in the slides needs advanced numeracy, let alone a background in statistics or econometrics.  The slides in the deck here are from a great Stuart McCoy/Marcelle Jansen presentation from the IoF Insight special interest group.

Analytical methods don’t need to involve statistical packages, and one of the points of the presentation is that we can gain significant insight using Excel and other widely-used packages.  In my example, I create a simple summary spreadsheet with weighted measures (scored at 1-10) of affinity and capacity.  These are pulled through into the first tab to create a score, indicating overall likelihood to give at a major level.  Some of the major indicators of affinity and capacity mentioned in the slides are:

  • Giving tenure/Continuity of Giving: the length of time a donor has been donating to your organisation, and how continuous this giving has been.  Continuous giving is great, but a high hit rate can also be really useful to measure
  • Giving ‘velocity’: basically the uplift in giving.  Dividing the current years total giving by the average of the previous three is a good way of doing this; another method is called ‘Compound Annual Growth Rate’ (CAGR)
  • Recruitment date: date of first contact with your organisation.  Interesting to contract this with the first gift date
  • Response ratios: rule of thumb is that people responding to more than one in 10 of your appeals is pretty engaged.  This is simply the total number of appeals divided by the number of responses
  • Unprompted communications: how often are supporters contacting you with being prompted?  Updating changed addresses, responding to surveys or questionnaires, signing petitions…all good signs of engagement
  • Wealth flags: setting up alerts for equity sales, or first-time donors who work with wealth managers are just two examples of how screening can be part-automated to help discovery through analytics
  • First gift amount: big first gift amounts are always to be followed up on
  • Current Lifetime Value (LTV): a great measure of potential and engagement, and very simple to calculate
  • Event participant/volunteer: again, simple to measure and a really strong signal of affinity and connection

More advanced methods of analytics include:

  • Regressions: these are a family of mathematical methods which aim to discover how important given variables are in a given situation
  • Text analytics: this uses software to scrape websites to carry out ‘sentiment analysis’, ie how users feel about a product or topic
  • Algorithms: an algorithm is a mathematical model to represent the relationship between variables
  • Automated scoring and screening: using business rules to automate database processes of screening
  • Machine learning: a branch of Artificial Intelligence which aims to teach computers to recognise logic, humour or other complex concepts

And before we get to advanced methods, a few resources to get prospect researchers started in using analytics:

Kevin MacDonnell’s blog: Cooldata

His and Peter Wylie’s 2014 book ‘Score!’ (ISBN 0899644457)

Josh Birkholz: Fundraising Analytics

See list at: https://www.worldcat.org/profiles/BenRymer/lists/3257763

Join Prospect-DMM: scarily advanced at times but well worth it: https://mailman.mit.edu/mailman/listinfo/prospect-dmm

Twitter: @joshbirkholz @iofinsight @n_ashutosh, @mpellet771, @mueggenburg

Finally: some potential pitfalls for those of us looking to use more analytics.  The main ones in my mind are:

  • Just because you find the answer you want(ed) doesn’t mean it is the right one.  Correlation does not equal causation.
  • Various factors quoted by Kevin MacDonnell and Peter Wylie in their great book ‘Score!’: “conservative nature of our institutions, a natural preference for intuition and narrative over data and analysis, a skills shortage, a fear of disruptive change, scepticism over the claims made for algorithms and a lack of time and resources”
  • A popular method in non-profit donor analytics is called ‘recency, frequency and value’ (RFV for short).  For me, this is part of the solution in understanding who is engaged with you organisation, but often leads back to those giving regular gifts by direct debit.  RFV therefor gives important insights, but is not the whole picture
  • With data, it is still true that ‘garbage in, garbage out’.  Take care of your data!  It will pay you back
  • Complex maths ≠ better results! There is no substitute for your expertise, judgement and attention

And the final word to MacDonnell and Wylie, who give a great summation of why prospect researchers should move into analytics as soon as possible:

“Data analysis is a rewarding, challenging, and above all fun line of work that will provide much value to your employer and a stepping stone in your career in fundraising to you”

Marianne Pelletier interview: “We assume we don’t share prospects, but of course we do”

/ rymerben / 2 Comments

Marianne

Marianne is Senior Consultant at Cornell University, and formerly worked at Carnegie Mellon and Harvard Universities.  She is a leader in advancement services, donor modelling and data mining and understanding donor engagement, speaking regularly at conferences and seminars on these subjects.  She tweets at @mpellet771.

A few points from the interview:

The use of insight can have powerful effects, increasing income and allowing not-for-profits to build stronger supporter relationships.  In the UK, prospect research has traditionally involved less quantitative or statistical methods.  However, ‘prospect research’ is different in the US, where it is largely data-driven.

Wealth screening: we all know it and use it.  And yet, even vendors admit that their information only covers around half of the millionaires in the population (and that total is probably an underestimate).  So, here will be a significant portion of the HNWI population whom charities are not aware of, sitting on their databases.  If not-for-profits modelled and analysed the level of wealth in more detail, they would almost certainly raise more form these groups.

Social media is coming to the fore in gaining valuable, ‘soft’ information on supporter preferences and interests.  Marianne’s team includes a full-time person scraping information from the web (and hand-connecting this to relevant supporter records), including network information, which is mapped in NodeXL.  Text analytics is also in vogue.

The web has fundamentally changed customer care, and Marianne describes some of the key ways in which this has happened.  First, Amazon “spoiled it for us” by raising the bar for the level of customer service users now regularly expect.  Next day delivery, automated, ‘you might like’ suggestions, and hugely responsive customer service are now all par for the course, whereas before they were considered exceptional.  Charities must keep up with these developments or be left behind.

There is lots more in the interview — I hope you enjoy it.

Clara Avery interview: “Where I’ve seen consistent success is where we’ve done the basics right”

/ rymerben / 1 Comment

Clara A

Clara is Head of Supporter Insight and Development at Macmillan Cancer Support.  She joined Macmillan in 2003 and previously led their Direct Marketing and Sales teams.  She tweets at @claraavery.

Macmillan have been oneClara_Macmillan insight flow of the success stories of British fundraising in recent years, and Clara sets out why that is from her perspective. The diagram here sums up the process.  Evidence is required at each stage, from identifying the gap to assessing whether further investment is needed or the initiative has been successful.  Clara stresses in the interview that the challenge is not to never fail, but to make failure cheap, and to learn as much as possible from these ‘failures’.  As G. K. Chesterton said, “If it’s worth doing, it’s worth doing badly”.

Two things in the interview struck me as particularly significant:

First, decisive leadership.  Macmillan embarked on a “massive” restructure and investment programme in 2008, when many charities were cutting costs in the midst of the recession.  The results speak for themselves, with Macmillan’s voluntary income increasing by something like 50% since then.

Second, the importance of renewal.  The World’s Biggest Coffee Morning is now a byword for fundraising success.  But it really took off once Macmillan looked at what and who drove the event.  By integrating insight into the campaign it became what it is today.

However, we also talk about how the prospect research and insight teams are structured at Macmillan, how to understand donor motivations, and what one piece of information Clara’s team would like to have to raise more money.  Hope you enjoy the interview.