Assume for the sake of argument, however, that we have got all of these data sets, and we have agreed some rules for philanthropy, so our philgorithm can be set loose to analyse data on need, social impact and past instances of philanthropy. Is our work done? Not quite, from what I can see. We still need a way of assessing which of the approaches to philanthropy or donation strategies the philgorithm proposes (or adopts, if it is acting directly) is “better”. Hence we have to establish some sort of metric. Since we are assuming that we have data on outcomes, do we just look at which approaches deliver the best outcomes? We could, but one problem is that we would have to compare outcomes across totally different cause areas (or “impact verticals” as I was informed the other day they are now to be called. Although not by me anytime soon…)
We could, of course, make our philgorithm domain-specific i.e. we would stipulate one or more cause areas to focus on and the algorithm could then determine how best to allocate resource to address those causes. This would introduce an element of human involvement back into the process (at least up front), which obviously takes us further away from the goal of a truly independent philgorithm but also has the advantage that it might be more palatable in practice.
At least initially, donors are likely to want to retain the ability to set certain criteria within which any proposed philanthropy algorithm can operate- very few will be comfortable painting themselves entirely out of the picture.
This is definitely likely to be the case in the early days of developing AI philanthropy, as very few donors (or indeed advisors) are likely to be comfortable painting themselves entirely out of the picture. They will almost certainly want to retain the ability to set certain criteria within which any proposed philgorithm will operate.
This may simply be the choice of cause area, as suggested above, but it might also include other preferences that shape the eventual nature of the algorithm: eg a preference for social change advocacy or direct service provision, a preference for small/local organisations over large/national ones, a preference for social enterprise over traditional charity etc.
Obviously imposing criteria like these would make the philgorithm far more subjective, as it would no longer be able to determine the optimal approach based solely on an objective determination of what would deliver the best outcomes, but it would make it a better reflection of the reality of philanthropy as it is now. Donors might also be able to “toggle” the degree to which a philgorithm was based on purely objective criteria or on subjective ones; and it is possible that over time their behaviour would shift towards embracing objective criteria to a greater degree (or perhaps it wouldn’t – I think this is a fascinating thought experiment!)
If we don’t want to introduce any element of subjectivity or cause-area specificity into our philgorithm, then we need to look for an objective measure not just of which interventions are better than others within a given domain, but which causes themselves are “better” than others. And that brings us back to Effective Altruism, because it is currently the only attempt to do precisely that.
So developing a philgorithm to identify the best ways to allocate resources to the most effective interventions in order to address the most pressing needs, based on Effective Altruism principles and trained using data on previous instances of human philanthropy, might be a possibility. But it would still be subject to the criticisms levelled at EA that we touched on above. And unless we believe that EA is a credible general prescription for philanthropy as a whole (which I don’t happen to think it is), what we have designed is an EA machine learning algorithm rather than a truly general purpose philgorithm.
The question is: is that as far as we can go…? Perhaps not, as we shall see in the next section.
3 REINFORCEMENT LEARNING
Could we carry the AlphaGo analogy one step further, and use unsupervised reinforcement learning to create the equivalent of AlphaGo Zero for philanthropy? I.e. rather than feeding our philgorithm data on previous instances of philanthropy and supervising it to rate them in terms of EA principles, could we let it operate simply on the raw data about need and social impact, give it a metric against which it could score itself and then run vast numbers of simulations to determine which philanthropy strategies came out most highly?
The exciting possibility, as with Go, is that a philgorithm developed in this way might come up with approaches to philanthropy that we have never seen before or never could have thought of. (Although that might also be a cause for concern, as we shall see!)
A philgorithm developed via unsupervised reinforcement learning might come up with approaches to philanthropy that we have never seen before or never could have dreamed of.
The absolutely fundamental question here is what the goal of the algorithm is. That will entirely determine how it learns and the approach to philanthropy it develops, so there is an enormous amount riding on it. Again, we might take as our starting point Effective Altruism. But this (as I have argued) is just one approach, and if it is posited as a system of rules for philanthropy as a whole, those rules are clearly normative rather than descriptive (i.e. they represent a view about how things should be rather than an attempt to capture how things are). Using reinforcement learning might enable us to come up with an algorithm that applied EA in entirely novel and intriguing ways to come up with philanthropy strategies, but this would still be an EA-specific philgorithm rather than a general-purpose one.
It should be apparent at this point that if we are going to come up with a general purpose philgorithm at any point in the future, the real work to be done is not so much in terms of the computer science or programming required to design the algorithm, but rather in terms of the philosophy, political science, economics and so on required to give us a theoretical framework to describe what philanthropy is for and how it works that is broader than Effective Altruism. This will then enable us to define what the goal of our reinforcement learning algorithm should be. It should also be obvious that I will have to leave that question unanswered, as it is beyond my capabilities at this point to come up with an entire philosophical system for philanthropy (and certainly not within the context of an already over-long blog!)
If anyone wants a starting point for how we might go about deciding what the goal of our future philgorithm might be, there are places to look. I would recommend the work of the philosopher Nick Bostrom, for example. In his book “Superintelligence”, which is primarily concerned with the question of how we might design a general purpose AI without succumbing to the existential threat that it would simply destroy or enslave the human race (so it is a jolly old read…), he touches on various ways that we might design algorithms to “maximise good” from the point of view of human beings. This includes things like taking a naïve view (i.e. literally saying “maximise good/happiness/quality of life” and helping the algorithm to refine its understanding of what this means), or allowing the algorithm to determine for itself what constitutes “good” (with some appropriate human involvement, one assumes!)
The choice of goal for any future philgorithm will be crucial: if we get it wrong we may end up with significant negative impacts on charities and their beneficiaries through unintended consequences.
The other thing that Bostrom talks a lot about is the danger of unintended consequences when choosing a goal for AI development. Whilst his concern is general, the same thing applies to the specific case of philanthropy. That is why the choice of goal for any proposed future philgorithm will be so crucial: because if we get it wrong it will not necessarily just be a case of having a less effective version of automated giving than we might like – we might end up with significant negative impacts on charities and their beneficiaries as a result of perverse interpretations of well-meaning instructions. To give you a sense of what I mean, I will leave you with a few examples. Some are obviously quite far-fetched, but they are designed to illustrate the point. Also, once you start playing my new parlour game of “Suggest unintended consequences of AI philanthropy”, you’ll see that it becomes quite easy to get carried away…
SCENARIO 1:
Goal : “Maximise Good Outcomes”
Unintended consequence: AI decides to redefine criteria of “good” or what constitute relevant outcomes to make it easier to maximise performance based on existing strategies.
SCENARIO 2:
Goal : “Maximise the number of Quality Adjusted Life Years (QUALYs) for the greatest number of people”
Unintended consequence A: AI falls victim to “Pascal’s mugging” i.e. deciding that resources should be directed toward an event with a vanishingly small probability but an enormous impact on human lives if it were to occur. E.g. collision with an extra-terrestrial object that might cause a mass extinction event.
Unintended consequence B: One interesting twist on this scenario is if the AI determines that the biggest risk to humanity comes from a future technological singularity (i.e. when machine intelligence accelerates past the point where humans would ever be able to catch up). It might therefore think that the best strategy is to put all of its resources into finding ways to undermine advances in AI research and technology i.e. it would direct its energies at ensuring its own development was stifled.
SCENARIO 3:
Goal: “Make everyone happy”
Unintended consequence : AI puts all of its resources into pharmacological and neurological research to design a substance with no physical side effects that induces a constant state of euphoria, and then designs mechanisms to ensure that the maximum number of people get access (e.g. by addition to water supply)
SCENARIO 4:
Goal: “Maximise average human happiness over time”
Unintended consequence A: AI determines that the major problem facing the world is overpopulation, and that actually by reducing the population to a lower level, the average amount of happiness can be dramatically increased. Hence it develops a strategy that includes taking money away from prevention for certain diseases, actively funding anti-vaccination propaganda and promoting lifestyle choices that lead to known health problems whilst also lobbying for changes to healthcare that would reduce the chances of people with certain conditions surviving.
Unintended consequence B: Or, to take this to the extreme, perhaps the AI decides that the best way to maximise average happiness is to concentrate its efforts on making a very small number of people happy to an extreme degree for as long as possible. Hence it develops euphoria-inducing substances (as above) and gives these to a small cohort of humans kept in laboratory conditions, whilst using life extension technologies to keep them alive as long as possible (perhaps even in perpetuity). So, over time, the extreme level of happiness of this cohort would ensure a higher average, even if the rest of the population lived in abject poverty and misery.
As I acknowledged, these examples are merely thought experiments (and ones of an increasing dystopian nature, obviously!) The point is not to suggest that these are in danger of happening now, or indeed at any point in the future: rather it is to make it clear that the process of designing algorithms to deliver social good could have significant unintended negative consequences so we need to be very careful how we proceed. There is a great deal of work still to be done in developing the philosophical, political, social and psychological frameworks around philanthropy; because what are currently intended as efforts to describe what philanthropy is now could unwittingly become criteria that determine what philanthropy can be in the future if they are enshrined in code that governs how it operates. This means that the stakes are potentially very high.