Economics 13: big data

by George Hatjoullis

The discussion so far has been limited to ‘macro economics’. This addresses economic society in aggregate. However, society is made up of individuals and aggregates are the sum of each action. The reason two branches of economics, macro and micro, have evolved separately is because of the fallacy of composition. Sensible individual behaviour does not automatically translate into sensible aggregate behaviour. It may be sensible for each individual to save more but if this increases aggregate net saving it may, as we have seen, push GDP below potential. We will return to macroeconomics after this brief flirtation with microeconomics.

Microeconomics concerns itself with, among other things, individual consumer behaviour. The individual has wealth and income (could be negative or zero) and must deploy this to purchase goods today. Economic theory uses various constructs to prove an upward sloping demand curve for any particular good. The higher the price, the less the individual will wish to spend on a particular product, other things being equal. These other things include wealth, income, the price of other goods, quality and anything else that might be relevant to the purchase. Economic theory does recognize that this negative relationship between quantity demanded and price might not apply in some circumstances e.g. if price becomes associated with quality or status. However, the basic assumption is that an upward sloping demand and price curve applies.

A great deal of applied economics is devoted to estimating demand curves for particular products. The work has historically focused on estimating the demand curve for the product (aggregated across all consumers) and not an individual’s demand curve for a particular product. The reason is that it is difficult to conduct an experiment and observe how an individual varies his demand as price varies. At least it was difficult. In these days of big data it is less difficult, and certainly possible.

The big data project in progress is collecting vast amounts of information on individuals. Some may be collected in anonymous form but need not be. For example, loyalty cards collect a great deal of individual behavioural data. The appearance of coupons discounting some product is not arbitrary. The allocation of coupons has been derived from an analysis of collected data and can be personalized. Not everyone necessarily gets the same coupons. Internet users give even more behavioural data to the firms and the state. Deriving an individual’s demand curves is no longer impossible.

This raises an interesting and somewhat disturbing point. The late Professor Ivor Pearce, then of Southampton University, introduced me to the search for an integrating factor for demand curves*. This seems to have been his major contribution to economic theory and he was evidently proud of his work. The demand/price curve can be thought of, mathematically, as the first partial derivative of a consumer reaction function (a utility function is an example) in which price appears as an argument. This derivative tells us how quantity demanded would vary for small changes in price, if other arguments in the reaction function are kept constant.

If we can observe an individual’s demand curve then it is in principle possible to derive the form of the reaction function by integrating the demand curve. In layman’s language, observing how we respond to price changes for turnips can reveal a lot more about our individual behaviour than just our attitude to turnips. Professor Pearce derived some interesting but limited restrictions on the form of the integrating factor. He would have been excited by the possibilities offered by big data.

The big data exercise collects a lot more information on individuals than merely their response to small changes in price. The exercise effectively provides a numerical solution to the problem of integrating the demand function. This is presumably what the big data project is all about. It is about identifying an individual’s reaction function. It is about forming a usable and detailed functional form that describes individual behaviour. From this functional form firms (and governments) can predict, respond to and influence behaviour at the level of the individual and not merely the ‘average’ individual.

I was concerned as a student as to the philosophical and moral implications of the work of Ivor Pearce and others in this field. However, as a student the main preoccupation is with passing exams. The big data project reminded me of my student conversations and concerns and seems very relevant today. Should we seek to know so much about an individual’s reaction function? Should we allow this knowledge to be available and to whom? The potential for misuse is obvious. Psychology as a subject also concerns itself with an individual’s behaviour. However, the psychology profession is regulated and subject to strict ethical standards. There are safeguards. They may not be deemed adequate but they exist. It is unclear how this big data project is being regulated and what safeguards are in place, let alone if they are effective. I wonder if all those that give their data freely realize what they are giving away.


A Contribution to Demand Analysis 1964