Select Page

How macroeconomics can push forward the frontiers of Data Science

Sep 12,2017 | Modeling

Our goal is to convince you that the practice of Data Science today can learn something from macroeconomics of the 1970s.

The Takeaway

In 1976, Robert Lucas critiqued macroeconomic forecasting by arguing that models based on aggregate data needed to be replaced with ones estimated from individual-level choices. Data Science today needs to evolve in a similar fashion.  Three practices would help:

  1. Move beyond the Business Intelligence style of correlation-based analysis
  2. Make decisions based on individual user behavior, rather than population averages
  3. Update analyses in real-time

Macroeconomics of the 70s was a field very much in flux – forecasting growth, unemployment, and the effects of monetary policy were pressing issues; aggregate data on economic conditions was getting more accurate and extensive; and better computers allowed more and more complex regression analyses to be performed.  But the predictions of these statistical forecasts proved unreliable for policymakers.  As a famous example, the Phillips curve found a negative historical correlation between unemployment and inflation – high inflation tended to coincide with periods of low unemployment, and vice versa.  But when central banks reacted by pursuing inflationary policies, unemployment didn’t drop!  

Future Nobel laureate Robert Lucas provided an explanation for this sort of forecasting failure in a seminal 1976 paper which came to be known as the “Lucas critique”.  His argument was essentially that contemporary macroeconomic forecasting confused correlation with causation – high inflation today may be correlated with low unemployment tomorrow, but this doesn’t mean it causes it.  

Lucas argued that forecasting models had to be revamped.  Models estimating correlations between aggregate variables would be thrown out.  In their place would be models capturing individual-level economic choices, with data used to estimate the “structural parameters” governing their behavior.  These structural parameters would ideally be stable across time and environments, particularly following an intervention by a government or central bank.  Only then could forecasting models really become useful for predicting the results of policy changes.  

We think the data science world of today is facing many of the same challenges confronted by macroeconomists of the 1970s.  The field is overdue for a Lucas critique of its own to point the way toward a new generation of models providing better predictions and more successful and cost-effective interventions.

 

Data Scientists need to get to the atomic level

The world of Data Science has spent the last two decades on a path mirroring macroeconomists a quarter century before them.  As Business Intelligence tools have improved and hiring data scientists has become common practice, business analytics has grown into a sophisticated effort to find “actionable insights”.  In other words, what factors drive user behavior and how can companies use them to proactively engage with customers or create policies that drive desired outcomes?

This analysis — though often sophisticated in execution — is conceptually simple: for a desired outcome, are there user traits, either demographic (who is this person?) or behavioral (how has this person interacted with my product?), that describe a user’s likelihood to engage or purchase?  The standard approach uses broad characteristics of coarse demographic bins.  A typical Business Intelligence result from Online Advertising looks like this:  “Women aged 25-35 in California clicked on this ad at a rate 2x higher than Men aged 55+ in Wisconsin.”

Ubiquitous collection of demographic and behavioral data on users has made such insights much easier to obtain.  However, it has also made possible much more powerful analyses.  With data collection becoming ever more granular, we now have a substantial amount of “atomic-level” user data.  This data enables us to learn about individuals, and not just population averages.  Now we know not just that “Men 25-35 in California tended to buy at a rate of 0.1%”, but that our individual customer Erik Madsen “has bought a Plush Stuffed Animal every time he was shown an ad for one.”

This atomic-level data unlocks more tailored interventions.  (An “intervention” is any action a company takes to affect customer behavior.)  Now when we want to boost sales via promotion, instead of advertising a discount on our Plush Animals to “All Men aged 25-35 in California”, we can more precisely target “All Men aged 25-35 in California who have purchased our product at least 3 times in the last month”.  This benefit of more granular analysis is clear and well-understood.

A less obvious opportunity afforded by atomic-level user data is the design of interventions that avoid “attenuation.”  We use the term attenuation to describe a commonly observed discrepancy between predicted and actual effectiveness of an intervention.  In particular, models based on aggregate correlations in customer data will typically over-predict the success of an intervention.  This is because, as Bob Lucas pointed out, those correlations will usually weaken when the environment changes.  By contrast, when we model individual level behavior, we are less likely to confuse correlation for causation and more likely to identify patterns in user behavior that hold up after an intervention.  This strategy is more akin to modern structural macroeconomic modeling, in contrast to pre-Lucas critique forecasting based on aggregates.  In the ideal, atomic-level modeling resolves the problem identified by the Lucas critique.

Why is atomic-level analysis the answer?  Recall Lucas’s original prescription: anchor your analysis around behavioral patterns that will remain invariant across environments.  It’s true that user invariants in a Business Intelligence context will be more situation-specific than the universal invariants of Economics (e.g. “Consumers save more when interest rates rise”).  Even so, individual user traits are much more likely to be stable across time and environments than characteristics of an entire demographic segment.  User traits look more like structural parameters and less like aggregate correlations the more we drill down toward the atomic level.

 

Atomic-level modeling in action

A concrete example will help illustrate the concepts we’ve been discussing.  Consider a subscription media company, The Political Scientist, working to reduce churn.  Their analytics team starts by looking for characteristics of subscribers vulnerable to churn, and discovers that failing to sign up for auto-renewal significantly increases subscription lapse in the next cycle.  They propose that their marketing team offer discounts for signing up for auto renewal to increase customer retention.  

This isn’t a crazy proposal by any means.  Many companies offer such discounts, and their effectiveness lines up with results from behavioral science on the power of default options.  What could go wrong with making decisions from this sort of aggregate analysis?  An obvious shortcoming is that without any targeting, the scheme will be expensive – The Political Scientist will offer discounts to plenty of people who wouldn’t have churned even at full price.  More granular analysis can reduce this cost by identifying subscribers with constellations of characteristics indicating high churn with more confidence.  

But there’s an additional, more subtle attenuation effect that is just as important.  Suppose subscribers in our example have two relevant individual characteristics – “auto-renewal status” and “number of articles viewed on The Political Scientist website”.  It’s natural that subscribers may self-sort into auto-renewing based on how often they read The Political Scientist.  So consider the behavior of a subscriber who’s offered a discount to sign up for auto-renewal.  Subscribers who only read sporadically will be happy to take advantage of this deal.  But if they decide they’re not getting enough value out of the site, they may still go through the trouble of cancelling.  The correlation between auto-renewal and churn would then be attenuated once the marketing team offers a discount.  In other words, auto-renewal numbers may spike, without yielding the drop in churn predicted prior to the intervention.  Discounting could turn out to be not only expensive but ineffective at influencing the desired metric.  This is the Lucas critique in action in a Business Intelligence context.

Let’s see how atomic-level modeling improves the situation.  Suppose The Political Scientist marketing team tailors the intervention to target only subscribers who are also consistent users of  the website.  It’s likely these users will get enough benefit from the site going forward that they won’t bother cancelling once they’re signed up for auto-renewal.  Targeting discounts to this group will lead to a much less attenuated effect than would a mass intervention across the entire subscriber base.  In the language of the Lucas critique, drilling down to the atomic level helps pick out so-called “structural parameters” of user behavior which remain stable after an intervention.

 

Keeping up with how quickly your data is changing is keeping up with your business

Analyzing user behavior at the atomic level greatly improves the accuracy and cost-effectiveness of your models.  But it is not enough merely to cut data at the atomic level.  User behavior can change rapidly as business conditions evolve.  It is therefore necessary to track analytics “in real time” to keep up with these changes.

Here are two simple examples we’ve noticed:

  • An individual’s propensity to buy tickets for a sports team’s games depends on the team’s recent win/loss record.
  • A user’s willingness to pay for a ride-share company’s product depends on whether their competitor is having a promotion.

Marketing, sales, and client service professionals can rattle off myriad similar examples that litter their daily work.

Interestingly, though some of these shifts are observable, others are not.  Any sports team knows its win/loss record.  On the other hand, very few companies have data on their competitor’s marketing plan.  Regardless, both observed and unobserved events impact the effectiveness of interventions.  This illustrates the need to keep atomic-level models updated in “real time” to keep pace with changes in user behavior.  Otherwise stale analytics introduce another source of attenuation and dilute the gains from modeling at the atomic level.  If you’re not keeping up with how quickly your data is changing, you’re not keeping up with your business.

Erik Madsen

Assistant Professor of Economics, New York University

Anthony Chong

CEO/Co-Founder, IKASI

24 Comments

  1. Interesting article Anthony and Erik. I agree that analysis at the individual level, rather than the aggregate level, will yield insights beyond what group averages will tell us. I wonder if analysis at the individual level could also lead to re-categorization of demographics bins. e.g. I have to imagine that somewhere out in the wide world, there’s another person much like Erik who shares an insatiable need to purchase Plush Stuffed Animals (other than simply adding him to the the Men & Women aged 0-10 in California group).

    Reply
  2. What do you see this saying about the role of individuals / individuality in emerging of ‘demographics’? Can we really leave it to “demographics are phantoms that never really existed”? Or does each individual have some naive (or significant) sense of the system of demographic dimensions that they are immersed within? Such a sense could lead people to adjust themselves toward a position that reinforces or disrupts a demographic trend.

    Reply
  3. Doctors have many challenges to face as they are perennially surrounded by patients, diseases, hospital duties and over-extended or odd shift timings. Universally, doctor is considered to be a noble profession and respectable one at that, but a doctor also has to work under immense pressures, emotional strains and other physical challenges.

    A regular physician like most of us at some point face will have to deal with personal situations such as important family affairs, family holidays, sickness or pregnancy that may force them to abandon medical duties. At the same time, a hospital or a healthcare facility is also constantly faced with emergency situations that demand all hands on deck round-the-clock. Therefore, every hospital, clinic or nursing home is compelled to hire locum tenens or substitute doctor in order to keep the staffing under control at all times.

    In fact, locum doctors are the most valuable asset for the medical community because they provide quality medical care and act as a helping-hand in emergency situations when the medical facilities need them the most.

    Unlike regular or permanent doctors, locum doctor jobs are also ideal career options for medical interns and graduates because they offer a wide array of medical exposure in varied medical specialties, work cultures and healthcare systems. Locum jobs are challenging and flexible, thus an increasing number of medical professionals have benefitted from these jobs, so whether one is looking for a family physicians position or in a hospital or in a clinic, locum jobs for doctors are available at all levels and in different healthcare systems.

    In addition, being a locum doctor gives a medical professional the control over their working hours, location of work and choice of area of specialisation. Technically, locum positions are not restricted to general physicians but they are also extended to other fields of medical specialisations such as cardiology, neurology and many more.

    Travelling can be an integral part of locum jobs, and these distinctive features are a boon for many dedicated medical professionals who are eager to expand their medical careers with loads of multi-cultural medical experiences. The fact that locum agencies in the UK recruit tens of thousands of locums from across the globe in various NHS hospitals, private clinics, nursing homes and other public hospitals speaks volume of the popularity of locum jobs.

    Locating or getting a locum tenens job is a simple task as long as you are registered with one of the many reputable locum agencies. These agencies act as the middle man between locum tenens and medical facilities, and they also look after all the details pertaining to travel for locum tenens, accommodation and the nature of locum work.

    Thus, maintaining a healthy locum doctor-agency relationship benefits both the parties, and it also increases the probability of getting recommendable employment opportunities and businesses or vice-versa.

    Reply
  4. Trying To Find The Best Diet Pill?

    Trying to find the best diet pill may seem like an impossible task, especially with the multitude of diet pills available for purchase. Many people purchase a diet pill only to find out that the pill makes them feel jittery, nervous, or often has no effect at all.

    Diet pills frequently contain the same or similar combination of ingredients and rarely contain anything new, innovative, or undiscovered to the supplement / weight loss industry. So, how can you find the best diet pill when most diet pills are made with similar ingredients?

    One of the most common problems associated with taking diet pills is that the person taking the diet pill is uneducated about the dosage, effects, and promises offered as they relate to each diet pill. The research at website finds that there are three factors that should be taken into consideration when deciding to take a diet pill.

    Dosage:
    It is important to take the pill exactly as recommended on the product label. Some people choose to increase the dosage thinking that the product will work faster or better. This is not the case, and many people become sick in response to the large dose. Reviewers at website often suggest that the recommended dosage be cut in half to give the body time to adjust to the stimulant in the diet pill. After the body has adjusted, it is fine to begin taking the regular dosage as recommended on the product label.

    Effects:
    The effects listed on the product label are there because these are the effects that the product has had on ‘some’ of the test group. Some of the diet pill testers may be fine taking the product, while others may have adverse effects. The diet pill companies print this information to educate the buyer as well as to protect themselves from lawsuits. The consumer needs to read the label and educate themselves before taking the product. Many people who are sensitive to caffeine are surprised when the diet pill makes them feel nervous or nauseous, but this information is likely printed on the product, so with a little research these affects can be avoided.

    Promises:
    If you read the fine print on product claims for diet pills and other weight loss supplements, you will see ‘results not typical’ printed very small somewhere where you are not expected to look. The diet pills advertised on television are responsible for some of the most outlandish claims. The results claimed in these advertisements are often unattainable within the given amount of time outlined in the ad. Don’t expect to see results in two weeks like a lot of ads claim.

    Wouldn’t it be great if you could read reviews for diet pills from actual users of each diet pill? Diet Pill Reviews website has taken the trouble out of searching for the best diet pill. You can read reviews of over 150 of the most popular diet pills available.

    Copyright 2006, Diet Pill Reviews viagra pas cher

    Reply
  5. I want to point out my appreciation for your kindness supporting visitors who actually need assistance with this particular issue. Your personal commitment to passing the solution all-around appears to be quite advantageous and has in most cases enabled regular people like me to get to their objectives. Your new informative publication denotes a great deal a person like me and additionally to my colleagues. Regards; from all of us.

    Reply
  6. Shops that require protection for high risk goods held on the premises will usually need to declare the total values of each stock item. High risk shop stock and goods are those that attract thieves and are expensive to replace. Examples of high risk stock items are electronic equipment, cigarettes, and tobacco, designer clothing, computers and digital equipment, software, computer games and consoles, drugs pharmacy and medicines, watches and jewellery, mobile phones and radios, photographic equipment, power tools, TVs, DVDs, CDs and Wines and Spirits.

    If your shop has high risk stock you can reduce the cost of your premiums by having adequate security in place. This includes an insurance company approved burglar and fire alarm, window grills, shutters and bars, CCTV and sprinklers. Many shop insurers will only offer stock cover if the minimum levels of security are in place for all shops, regardless of the stock contents held. A lot of insurers may offer further large discounts to the premium if the shop owner lives on or above the premises and is there at night.

    Shops by their very nature deal with members of the public and a good insurance policy will usually contain liability cover as standard. This should include Public Liability of up to ?2,000,000 for any one claim by a member of the public who may suffer loss or injury visiting the shop.

    If you employ staff all policies will offer Employers Liability cover of up to ?10,000,000 one event and because shops sell goods and services, Products Liability cover of ?2,000,000 for any one period of insurance.

    https://www.cialissansordonnancefr24.com/achat-cialis-sans-ordonnance/

    Reply
  7. ome people, especially those running on busy daily schedules tend to use the pills to help maintain weight since they can not afford to follow all the diet programs. This is not advised. It is recommended that one seek advice from a professional in this field before using the pills. This can save one from many dangers associated with the misuse.

    The diet pills should always be taken whole. Some people tend to divide the pills to serve a longer period of time. This is not advised and can lead to ineffectiveness. If it is required that one takes a complete tablet, it means that a certain amount of the ingredients are required to achieve the desired goal. It is also recommended that one does not crush the pill and dissolve it in beverages. Chemicals found in beverages have the potential of neutralizing the desired nutrients in the pill thereby leading to ineffectiveness. The best way to take the tablets is swallowing them whole with a glass of water.

    https://www.cialissansordonnancefr24.com/cialis-comment-le-prendre/

    Reply
  8. Thank you so much for giving everyone an extraordinarily breathtaking opportunity to read articles and blog posts from this website. It can be very ideal and stuffed with a great time for me and my office fellow workers to visit your website at a minimum thrice per week to read the latest tips you have got. And lastly, we are usually happy with your breathtaking points you serve. Some 2 areas in this posting are unquestionably the most suitable I’ve ever had.

    Reply
  9. A lot of thanks for all your valuable labor on this website. Betty enjoys doing investigations and it’s really simple to grasp why. My partner and i know all concerning the compelling means you deliver very useful tips through the blog and as well welcome contribution from others about this matter while our favorite princess is really learning a lot. Take advantage of the remaining portion of the new year. You have been carrying out a first class job.

    Reply
  10. I and also my pals were analyzing the excellent advice from your web site and immediately came up with a horrible suspicion I never thanked the web blog owner for them. Those women are already for that reason happy to read them and have in effect really been taking pleasure in those things. Thank you for really being considerably helpful and then for making a choice on certain great information millions of individuals are really desperate to know about. Our sincere regret for not expressing gratitude to earlier.

    Reply
  11. I not to mention my guys happened to be viewing the good techniques from your web site while unexpectedly developed a terrible suspicion I never thanked you for them. My boys ended up stimulated to read them and have now without a doubt been taking advantage of them. Many thanks for turning out to be indeed thoughtful and also for finding these kinds of remarkable issues millions of individuals are really wanting to be aware of. My very own honest apologies for not expressing gratitude to sooner.

    Reply
  12. I precisely desired to thank you very much all over again. I am not sure the things that I would have made to happen in the absence of the actual solutions revealed by you directly on my theme. It has been a terrifying scenario for me personally, nevertheless coming across your specialized approach you treated that forced me to leap with delight. Extremely thankful for this assistance and as well , pray you recognize what a powerful job that you’re doing educating others with the aid of a web site. Probably you haven’t got to know all of us.

    Reply
  13. My husband and i have been quite satisfied Edward could do his survey while using the precious recommendations he had from your web pages. It is now and again perplexing to just happen to be making a gift of key points which usually other people could have been selling. Therefore we do understand we’ve got you to thank for this. The type of explanations you have made, the easy website menu, the relationships your site assist to engender – it is most remarkable, and it’s really aiding our son and our family do think the theme is satisfying, and that’s exceptionally fundamental. Many thanks for the whole thing!

    Reply
  14. I happen to be writing to let you understand of the awesome discovery my friend’s girl encountered reading the blog. She learned such a lot of issues, with the inclusion of what it’s like to possess an amazing giving character to get many others just have an understanding of a variety of impossible topics. You undoubtedly exceeded my expectations. I appreciate you for supplying the invaluable, dependable, educational as well as easy tips on your topic to Mary.

    Reply
  15. I have to point out my admiration for your kind-heartedness supporting persons who require assistance with the subject. Your special dedication to getting the solution up and down had been definitely practical and has encouraged individuals just like me to achieve their pursuits. Your entire invaluable key points implies this much a person like me and extremely more to my peers. Thanks a ton; from all of us.

    Reply
  16. Thank you so much for providing individuals with remarkably remarkable opportunity to check tips from this blog. It’s always very enjoyable and also jam-packed with a good time for me and my office acquaintances to visit the blog minimum thrice every week to read through the latest issues you have. And indeed, I’m also at all times impressed with your breathtaking hints you give. Certain 2 points in this posting are rather the very best we’ve ever had.

    Reply
  17. I have to convey my passion for your kindness for visitors who really want help on this important concern. Your very own commitment to getting the solution around had become particularly effective and has continuously empowered men and women like me to reach their aims. Your useful useful information means a whole lot to me and even more to my fellow workers. Thank you; from each one of us.

    Reply
  18. I happen to be commenting to make you know what a great experience my cousin’s daughter enjoyed reading through your site. She even learned plenty of things, most notably what it is like to possess an incredible coaching heart to make many more with ease comprehend specified hard to do subject matter. You really surpassed our own expected results. Thanks for supplying the informative, safe, explanatory as well as fun thoughts on your topic to Tanya.

    Reply
  19. Thanks for all your valuable effort on this website. Ellie delights in carrying out research and it’s really easy to see why. We all hear all concerning the dynamic method you make reliable tricks through the website and therefore encourage participation from other ones about this article plus our own princess is understanding so much. Have fun with the rest of the year. You’re the one performing a fantastic job.

    Reply
  20. I simply wanted to develop a remark to appreciate you for all the superb instructions you are giving out on this website. My time-consuming internet investigation has at the end been recognized with wonderful insight to go over with my company. I would point out that we readers are rather lucky to dwell in a really good site with many special professionals with helpful plans. I feel very blessed to have seen your entire web site and look forward to some more fabulous minutes reading here. Thanks a lot again for all the details.

    Reply
  21. I am writing to let you be aware of what a extraordinary experience my friend’s girl undergone viewing your site. She noticed several things, which included what it is like to have an ideal teaching character to make certain people easily fully grasp a variety of impossible issues. You actually exceeded our expected results. Thanks for coming up with those beneficial, trusted, informative not to mention fun tips about the topic to Emily.

    Reply
  22. I want to express my thanks to this writer just for bailing me out of this type of setting. After browsing through the the web and getting proposals which are not beneficial, I thought my life was well over. Being alive devoid of the answers to the issues you have resolved all through the article is a critical case, and those which may have negatively affected my entire career if I had not encountered your blog post. Your actual skills and kindness in playing with every part was valuable. I don’t know what I would’ve done if I had not come across such a solution like this. It’s possible to now relish my future. Thanks for your time very much for this high quality and result oriented help. I will not hesitate to suggest the sites to anyone who wants and needs recommendations on this subject matter.

    Reply
  23. I happen to be writing to make you understand what a notable encounter my wife’s child undergone checking the blog. She came to find a lot of details, most notably what it is like to have a very effective coaching heart to make other individuals with ease know just exactly specific hard to do topics. You truly exceeded readers’ desires. Thanks for rendering the valuable, dependable, educational and cool thoughts on the topic to Emily.

    Reply

Submit a Comment

Your email address will not be published. Required fields are marked *

Share This