Fifteen Eighty Four

Econometricians develop and use methods and techniques to model economic behavior, create forecasts, to do policy evaluation, and to develop scenarios. Often, this ends up in some advice. This advice can be a prediction for the future or for another sector or country, it can be a judgment on whether a policy measure was successful or not.

This book Ethics in Econometrics is inspired by my experience with consulting for companies and institutions, and by my experience with supervising hundreds of students in the past twenty or so years when they did their internships with companies. Basically, the key premise is that the advice of an econometrician is usually given to people who are not econometricians. Thus, the advisees must trust the quality of the advice. Such trust can be gained by making data available, by allowing access to the programming code, by writing and presenting in a non-technical manner, but then still, there remains to be a gap as non-econometricians cannot check all this. Econometricians therefor (must) make choices that can only be understood by fellow econometricians. The main claim in this book is that it is important to be clear on all those choices.

Hence, this book is about choices. More precisely, it will show how substantial their consequences can be, even though they may seem harmless in the beginning. Indeed, models can be useful and informative, but sometimes they are not. For example, it may not be too difficult to make forecasts look very reasonable and please the (paying) customer. If a client wants to hear that profits shall go up next year, it is not too difficult to create a model and an associated forecast, even using actual data, that meets those wishes. And, if someone wants to show that a policy measure was not successful, there are ways to make that happen, at least, on paper.

If you know how to play around with your data and models, the benefit is that you can also recognize when someone else did something wrong (by accident or on purpose) and then you can ask the proper questions. Indeed, we will encounter cases of scientific misconduct that could have gone unnoticed if the misconduct were conducted much better. And we will also see that some by now classic misconduct cases were obviously geared by a shortage of knowledge on basic statistics by the deceivers.

This book has thirteen chapters, and each chapter can be considered independently of what is or will be presented in other chapters. Most chapters contain various empirical exercises to illustrate the main points. All chapters contain detailed discussions of real cases, where the data are made available in Excel format. This allows for replication by the reader.

We start with what is commonly viewed as good ethical practice. There are various guidelines available for research and advice, both in general as well as for the application of statistical methods. Then we deal with a few cases of scientific misconduct, concerning creating fake data, fake tables with “results”, and fake statistical summaries of the data. We deal here with willful deception. But we also see that simple methods can detect misconduct. In fact, it is shown how scientific misconduct could have done “better” in the sense that the chances of discovery would have been smaller.

Broadly speaking, the rest of the book deals with all kinds of phenomena that one can encounter in practice, and the list is of course not complete. We will learn that econometric analysis can seriously be distorted by (highly) influential observations. And there do not have to be many of them. Also, they are influential in small samples and in large samples.

In practice it turns out that there is always more than one model that matches the data or  gives decent forecasts. We will argue that someone combining the results seems better than searching for the one and only.

It may also be that the first designed model may suffer from some shortcomings. Endogeneity, measurement errors, aggregation, and multicollinearity all can require that the model needs to be modified. This can be done in various ways, and we must be clear how we solved matters.

In practice it regularly occurs that data are missing. There is no need to panic, there are plenty of ways to address this, but we should be aware that these all have consequences for subsequent analysis. Even the simplest interpolation schemes can have dramatic effects.

The nightmare for all econometricians is a spurious relation. That is, you think there is something, but there is not. There are various reasons for such spuriousness, and there are various solutions. What also can happen is that certain data features spoil a good sight on other features. Levels shifts that make you think there is a trend, for example.

What is certainly relevant in research practice is that we should be clear on the limits to predictability. That is, how far ahead can we sensibly predict? And, if we feel that forecasts need to be modified manually, there are also a couple of guidelines that are helpful.

The book concludes with a current phenomenon when data become plentiful, well, in fact, that there are too many. Do we aggregate the data? Do we select observations from the data to make analysis easier? Do we cluster the data before we consider a regression model? Do we rely on the help of so-called machine learning algorithms? But then, can outcomes from algorithms be trusted? Are they the way forward for much empirical analysis in the future? What is and what will be the contribution of humans? And when racial or gender discrimination is at stake, is that to be blamed on the program code or on the programmer?