Wednesday, April 4, 2018

How today’s pros solve math problems: Part 3 (The Nueva School course)

By Keith Devlin

You can follow me on Twitter @profkeithdevlin


NOTE: This article is the final installment of a four-episode mini-series posted here starting in mid-January. In writing it, I have assumed my readers have read those three earlier pieces.

At the end of last month’s post, I left readers with a (seemingly) simple arithmetic problem. I prefaced the problem with the following two instructions:

1. Solve it as quickly as you can, in your head if possible. Let your mind jump to the answer.

2. Then, and only then, reflect on your answer, and how you got it.

The goal here, I said, is not to get the right answer, though a great many of you will. Rather, the issue is how do our minds work, and how can we make our thinking more effective in a world where machines execute all the mathematical procedures for us?

Here is the problem.

PROBLEM: A bat and a ball cost $1.10. The bat costs $1 more than the ball. How much does the ball cost on its own? (There is no special pricing deal.)

What answer did you get? And what did you learn from the subsequent reflection?

Before I continue, I should note that the use of this problem (which you can find in many puzzle books and on countless websites) in the context of trying to maximize the human mind’s innate abilities in order to become good 21st Century mathematical thinkers, is due to Gary Antonick, with whom I co-taught a Stanford Continuing Studies adult education course last fall. It was in that course that I gave the second iteration of the UPS problem I subsequently based my Nueva School course on. The discussion of the bat-and-ball problem that follows is the one Antonick presented in our course.

Now to the problem itself. The most common answer people give instantly to this problem is that the ball costs 10¢. It’s wrong (and many realize that is the case soon after their mind has jumped to that wrong number). What leads many astray is that the problem is carefully worded to run afoul of what under normal circumstances is an excellent strategy. (So if you got it wrong, you probably did so because you are a good thinker with some well-developed problem-solving strategies— problem-solving heuristics is the official term, and I’ll get to those momentarily. So take heart. You are well placed to do just fine in 21st Century mathematical thinking. You simply need to develop your heuristics to the next level.)

Here is, most likely, what your mind did to get to that 10¢ answer. As you read through the problem statement and came to that key phrase “cost more,” your mind said, “I will need to subtract.” You then took note of the data: those two figures $1.10 and $1. So, without hesitation, you subtracted $1 from $1.10 (the smaller from the larger, since you knew the answer has to be positive), getting 10¢.
Notice you did not really perform any calculation. The numbers are particularly simple ones. Almost certainly, you retrieved from memory the fact that if you take a dollar from a dollar-ten, you are left with 10¢. You might even have visualized those amounts of money in your hand.

Notice too that you understood the mathematical concepts involved. Indeed, that was why the wording of the problem led you astray!

What you did is apply a heuristic you have acquired over many financial transactions and most likely a substantial number of arithmetic quiz questions in elementary school. In fact, the timed tests in schools actively encourage such a “pattern recognition” approach. For the simple reason that it is fast and usually works!

We can, therefore, formulate a hypothesis as to why you “solved’ the problem the way you did. You had developed a heuristic (identify the arithmetic operation involved and then plug in the data) that is (a) fast, (b) requires no effort, and (c) usually works. This approach is a smart one in that it uses something the human brain is remarkably good at—pattern recognition—and avoids something our minds find difficult and requiring effort to master (namely, arithmetic calculation).

Of course, primed by the context in which I presented this particular problem, you probably expected there to be a catch. So, after letting your mind jump to the 10¢ answer, you likely took a second stab at it (or, if you were anxious about “getting a wrong answer,” made this your first solution) by applying an algorithm you had learned at school. Namely, you reasoned as follows:

Let x = cost of bat and y = cost of the ball. Then, we can translate the problem into symbolic
form as x + y = 1.10 ,   x = y + 1

Eliminate x from the two equations by algebra, to give
1.10 – y = y + 1

Transform this by algebra to give
0.10 = 2y

Thus, dividing both sides by 2, you conclude that
y = 5¢.

And this time, you get the correct answer.

You may, in fact, have been able to carry out this procedure in your head. When I was at school, I could do algebraic manipulations far more complicated than this in my head, at speed. But, truth be told, since I started outsourcing arithmetic to machines many decades ago, I have lost that skill, and now have to write down the equations and solve them on paper. (This is a confirmation, if any were needed, that arithmetic calculations do not come naturally to the human brain. Over the years, as my mental arithmetic skills have declined, my pattern recognition abilities have not diminished, but on the contrary have dramatically improved, as I learned—automatically, through exposure—to recognize ever more fine-grained distinctions.)

Whether or not you can do the calculation in your head, it is of course entirely formulaic and routine. Unlike the first method I looked at (a heuristic that is fast and usually right), this method is an algorithmic procedure, it is slow (much slower than the first method, even when the algebraic reasoning is carried out in your head), but it always works. It is also an approach that can be executed by a machine. True, for such a simple example, it’s quicker to do it by hand on the back of an envelope, but as a general rule, it makes no sense to waste the time of a human brain following an algorithmic procedure, not least because, even with simple examples it is familiarly easy to make a small error that leads to an incorrect answer.

But there is another way to solve the problem. It’s the way I addressed it, and, according to Antonick, who has given it to many professional mathematicians and asked them to vocalize their solutions, the way many math pros solve it. Like the first method we looked at, it is a heuristic, hence instinctive and fast, but unlike the first heuristic method, it always works.

This third method requires looking beyond the words, and beyond the symbols in the case of a problem presented symbolically, to the quantities represented. Though I (and likely other mathematicians) don’t visualize it quite this way (in my case it is more of a vague sense-of-size), the following image captures what we do.

http://www.stanford.edu/~kdevlin/

As we read the problem, we form a mental sense of the two quantities, the cost of the ball-on-its-own and the cost of the bat-plus-ball, together with the stated relation between them, namely that the latter is $1 more than the former. From that mental image, where we see the $1.10 total consists of three pieces, one of which has size $1 and the other two of which are equal, we simply “read off” the fact that the ball costs 5¢. No calculation, no algorithm. Pure pattern recognition.

This solution is an example of Number Sense, the critical 21st Century arithmetic skill I wrote about in the January 1, 2017 Huffington Post companion piece to the article I published on the same day as my article about all my math skills becoming obsolete, which I referred to in my last post here on Devlin’s Angle.

It is, I suggest, hard to imagine how a computer system could solve the problem that way. (Of course, you could write a program so it can perform that particular pattern recognition, but the essence of number sense is that you can apply it to many numerical problems you come across.)

Those three ways to solve the bat-and-ball problem I just outlined are examples of what the famous Australian (pure) mathematician Terrence Tao has called (in his blog), respectively, pre-rigorous thinking, rigorous thinking, and post-rigorous thinking. You can also listen to him explain these three categories in a short video in the Numberphile series.

Post-rigorous heuristic thinking is how today’s math pros use mathematics to solve real-world problems. In fact, as Tao makes clear, post-rigorous thinking is what the pros use most of the time to solve abstract problems in pure math. The formal, symbolic, rigorous stuff comes primarily at the end, to check that the solution is logically correct, or at various intermediate points to make those checks along the way.

In the case of solving real-world problems, the pros almost always turn to technology to handle any algebraic deductions. In contrast, though pure mathematicians sometimes do use those technology products as well, they often find it much quicker, and perhaps more fruitful in terms of gaining key insights, to do the algebraic work by hand.

So, one of the big question facing math teachers today is, how do we best teach students to be good post-rigorous mathematical thinkers?

In the days when the only way to acquire the ability to use mathematics to solve real-world problems involved mastering a wide range of algorithmic procedures, becoming a mathematical problem solver frequently resulted in becoming a post-rigorous thinker automatically.

But with the range of tools available to us today, there is a good reason to assume that, with the right kinds of educational experiences, we can significantly shorten (though almost certainly not eliminate) the learning path from pre-rigorous, through rigorous thinking, to post-rigorous mathematical thinking. The goal is for learners to acquire enough effective heuristics.

To a considerable extent, those heuristics are not about “doing math” as such. Rather, they are focused on making efficient and effective use of the many sources of information available to us today. But before you throw away your university-level textbooks, you need to be aware that the intermediate step of mastering some degree of rigorous thinking is likely to be essential. Post-rigorous thinking is almost certainly something that emerges from repeated practice at rigorous thinking. Any increased efficiency in the education process will undoubtedly come from teaching the formal methods in a manner optimized for understanding, as opposed to optimized for attaining procedural efficiency, as it was in the days when we had to do everything by hand. See Daniel Willingham’s excellent book Why Don’t Students Like School? for a good, classroom-oriented look at what it takes to achieve mastery in a discipline.

Now to that UPS routing problem that was the focus of my Nueva School course. [You will find it discussed here.] Here are some of the hints and suggestions about solving the problem I made to the students in the three courses where I used it. Whether they followed my advice was entirely up to them. The purpose of the course was not to solve the problem unaided—even an entire semester would not be enough time for that with students who had never approached a problem the way the pros do. Rather, it was to give them an experience of the method.

First, they had to work in teams of three to five. I let them select the teams, but said it would be good if at least one person on each team felt they were “good at math.”

Then, start out by using Google to find out what you can about the problem domain, and any attempts made by others to solve it.

Whenever you come across a reference to a concept, an approach, or a method that you suspect might be relevant, use general Web resources like Wikipedia to get an initial understanding of what they are and what they can do.

Follow any leads your search brings up to solutions of problems that look similar. Note what methods were used to solve them.

If you come across references to others who have worked on the problem, or a similar one, send them a brief email. You may not get a reply, but occasionally you will, and it could be invaluable. (I receive such emails all the time. Mostly I do not have time to respond, but occasionally one lands in my inbox when I have a spare moment, and I happen to know something that might help, so I shoot back a brief reply, often just a reference to a particular source.)

When you get to a point where you need to perform a specific calculation, perhaps because you have found a solution to a very similar problem someone else has obtained and published, but your data is different, use Wolfram Alpha. It is structured so you can use pattern recognition (of formulas) to identify the appropriate technique and then edit the example provided to be the one you want to solve.

Reinforce your use of Wolfram Alpha by using YouTube to find suitable videos that provide you with quick tutorials on the technique.

The resources I just mentioned are all listed on that chart of “Important Mathematical Technology Tools” I published with the first two articles in this series.

As it turns out, with the UPS routing problem, the sequence of steps I have outlined so far quickly leads to identification of a small number of possible solution techniques for which there are many very accessible YouTube videos, and in fact, for this problem there is no need to go much further into my list of tools, if at all.

You should, though, check out the various other resources on my list, to see what they offer. Each new problem has to be approached afresh, in its own terms. Twitter is on my list because it is my list, and I have sufficiently many math-expert Twitter followers that a quick tweet can often yield just the information I need, saving me having to send out a slew of emails to people I think might be able to help. LinkedIn is also idiosyncratic to me, since I have a good network of mathematics and technology professionals I can contact. But the other resources are pretty generic.

Ideally, everything goes much more smoothly if you can avail yourself the services of a math consultant to assist you in negotiating the various resources. (I was that consultant to the teams in the three courses I gave.)

Interestingly, in the final meeting of my Princeton class (which was the fist time I used the UPS problem in a course), after having the student teams present their solutions, I gave the solution I had obtained, at the end of which two individuals came up to me to say they hoped I had not minded their sitting in on the class. (It was an experimental course, and there had been strangers sitting in for one or two sessions throughout the semester, so I had not paid them any attention.) They were, they said, postdocs working with Professor X, who was a math consultant for UPS and had worked on the algorithm the class and I had been trying to reverse engineer. Hence their curiosity-driven attendance on the last day! Unbeknownst to me, my final lecture had been my oral exam!

“How did I do?”, I asked. “You got it pretty well right,” they replied.

Which was nice for me, but it would not have mattered if I had followed a different track. What was important from an educational standpoint was the process.

Something else I suggested to the class was to come up with a solution—any solution—as soon as you can. “Don’t worry if it is optimal or even right,” I said. “Just check it by computation, perhaps in the form of a spreadsheet simulation. Once you have some solution that you can check (in the case of my UPS problem, check against the shipping data I supplied, or any other UPS data you can find on the Web), you can iterate to find a better one. It might turn out that your first solution, or your first three or four, won’t even get you to first base, but in the process of formulating and checking those initial attempts, you will inevitably gain insight into the problem you are trying to solve. Remember, computation is cheap, fast, and essentially limitless.”

If you are not familiar with this way of solving math problems, it may not seem like an approach that will work. But it does. It is, in fact, how all of today’s pros do it!

If you have not already done so, now is a good time to check out the dictionary definition of the word heuristic! Here is Wikipedia’s (at the time of writing):

“A heuristic technique (from the ancient Greek for “find” or “discover”), often called simply a heuristic, is an approach to any problem-solving, learning, or discovery that employs a practical method not guaranteed to be optimal or perfect, but sufficient for the immediate goals. Where finding an optimal solution is impossible or impractical, heuristic methods can be used to speed up the process of finding a satisfactory solution. Heuristics can be mental shortcuts that ease the cognitive load of making a decision. Examples of this method include using a rule of thumb, an educated guess, an intuitive judgment, guesstimate, stereotyping, profiling, or common sense.”

Without an expert consultant, the heuristics approach to solving real-world problems can work, but it definitely goes a lot faster, and with a far great likelihood of success, if you have a math expert you can call on. Not to “do any math.” Computer systems handle those parts. Rather to help you negotiate the vast array of resources at your disposal and select the most promising one(s) to try next. For that is what using mathematics to solve a real-world problem really boils down to these days: managing resources.

And managing resources is something humans are innately good at. Natural selection always favors those creatures which are best able to manage the available resources. We are here as present-day humans because as a species we are good at doing that. What is new in the case of mathematical problem solving is that pieces of mathematics (formulas, equations, procedures, algorithms, techniques) are now among the “intellectual Lego pieces” (freely accessible on the Web) we can use as we assemble a solution.

As the students in my three courses could, in principle, attest, you don’t need vast expertise in mathematics to work this way. You just need to be a good thinker able to work in a small team. I say, “in principle,” since I think it highly likely most of not all the students felt they did not do much at all by way of using math to solve a problem. But that, I would say, is because they have a conception of “using math to solve a problem” rooted in the Nineteenth Century, if not the Fourth Century BCE. From my perspective, they absolutely were able to do what I just said they did.

Of course, they were not as good at it as I am. I’ve been at this game a lot longer, and, make no mistake about it, experience counts. (I think it is close to being the only thing that counts.)

What does not count, at least to any extent even remotely approaching the prohibitive degree it used to, is the ability to “do the math.” You just need to be able to select (hopefully, with help from someone with experience) the right pieces from the available online mathematical smorgasbord, and stitch them together in an appropriate way.

This kind of problem-solving doesn’t feel like math (as we all came to love or hate), that’s for sure. In fact, it doesn’t even feel like work. Once they got into the swing of it, even the students who declared they were not good at math or did not enjoy it, found they were having a good time, working in teams in a creative, explorative way. For the fact is, properly approached, humans enjoy problem-solving. (That’s another consequence of natural selection— problem-solving, particularly group problem solving, is one of our species’ key survival advantages.)

In fact, another way to look at the recent revolution in how we “use math” to solve real-world problems, is that it has brought “using math” into the mainstream of human group activities we naturally find enjoyable. At heart, mathematical thinking is little more than formalized common sense. It always has been. Which means it is something we can all do. (In my 2000 book The Math Gene, I presented an evolutionary explanation for the human brain’s acquisition of the ability to do mathematics, which implied that mathematical capacity is in the human gene pool, and hence available for all of us to “switch on.”) What caused many people problems over the centuries was that, before we had technologies that could handle the formal symbol-manipulation stuff, the only way to employ our innate capacity for mathematical thinking was to train the brain to do those manipulations. But manipulating algebraic symbols with logical precision is most definitely not something our brains evolved to do. (Our early ancestors’ lives on the savannas did not present much by way of a need for algebra.) So we find it very hard. Only with great effort over several years can we train our brains to do such work. And even then, we are error-prone.

Incidentally, practically everything I have said in this article applies to the way 21st Century coders work. In coding as in mathematics, the days are long gone when it was all about writing thousands of lines of instructions. The modern-day mathematician’s Web resource MathOverflow (on my chart of useful math tools) was modeled on, and named after, the coding world’s StackOverflow. Both groups of professionals use heuristics. In today’s world, highly regarded math problem solvers and good coders have simply acquired a richer and more effective set of heuristics than the ones who are less highly ranked. And for the most part, developing heuristics is a result of reflective experience, not some innate talent.

And there you have it. The primary goal in 21st Century mathematics-education-for-all is the development of a good repertoire of heuristics.

I’ll leave you with a graphical summary of Tao’s categorization of the three kinds of mathematical thinking we can bring to problem-solving. I introduced this categorization above to provide a perspective on the three phases each one of us has to go through to become proficient mathematical (real-world) problem solvers. But it also provides an excellent summary of three historical stages of mathematical thinking as it has evolved over the past ten thousand years or so, from the invention of numbers in Sumeria, where the mathematical thinking of the time was accessible to all, through three millennia of formal mathematics development, where many people were never able to make effective use of it, and now into the third phase, where, because of technology, mathematical thinking can once again be accessible to all.

http://www.stanford.edu/~kdevlin/

To be sure, we do not know the degree to which people have to master rigorous thinking to become good post-rigorous thinkers. As I already noted, I don’t for a second imagine that stage can be by-passed. (See the Willingham book I cited.) But, given today’s technological toolkit, including search, social media, online resources like Wolfram Alpha and Khan Academy, and a wide array of online courses, it is absolutely possible to master most of the rigorous thinking you need “on the job,” in the course of working on meaningful, and hence motivational and rewarding, real-world problems.

This is not to say there is no further need for teachers. Far from it. Very few people are able to become good mathematical thinkers on their own. Newtons and Ramanujans, who achieved great things with just a few books, are extremely rare. The vast majority of us need the guidance and feedback of a good teacher.

What the inevitable transition to 21st Century math learning requires is that mathematics teachers operate very differently than in the past. The days where you need a live person to deliver information are largely over. Today, teaching is much more a matter of being a coach and mentor. To be sure, you can occasionally find such teaching on the Internet, but it works only if you can be one-on-one with that teacher. I expect there will be change, but I don’t expect an economy of scale. If I had to make a guess, I would predict that in due course you will find your (specialist) math teacher by going online to a Math-Teacher-Match.com website, where you will be paired with a practicing 21st Century math professional who spends part of each day coaching and mentoring students.


LABELS: mathematical thinking, problem-solving, rigorous thinking, pre-rigorous thinking, post-rigorous thinking, Terrence Tao, social media in mathematics

Friday, March 9, 2018

How Today’s Pros Solve Math Problems: Part 2

By Keith Devlin

You can follow me on Twitter @profkeithdevlin


CHANGE OF PLAN: When I wrote last month’s post, I said I would conclude the description of my Nueva School Course this time. But when I sat down to write up that concluding piece, I realized it would require not one but two further posts. The course itself was the third iteration of an experiment I had tried out on a university  class of non-science majors and an Adult Education class. This series of articles is my first attempt to try to describe it and articulate the thinking behind it. As is often the case, when you try to describe something new (at least it was new to me), you realize how much background experience and unrecognized tacit knowledge you have drawn upon. In this post, I’ll try to capture those contextual issues. Next month I’ll get back to the course itself.


We all know that mathematics is not always easy. It requires practice, discipline and patience,  as do many other things in life.  And if learning math is not easy, it follows that teaching math is not easy either. But it can help both learner and teacher if they know what the end result is supposed to be.

In my experience, many learners and teachers don’t know that. In both cases, the reason they don’t know it is that no one has bothered to tell them. There is a general but unstated assumption that everyone knows why the teaching and learning of mathematics is obligatory in every education system in the world.  But do they really?

There are two (very different) reasons for teaching and learning mathematics.

One reason is that it is a way of thinking that our species has developed over several thousand years, that provides wonderful exercise for the mind, and yields both challenging intellectual pleasure and rewarding aesthetic beauty to many who can find their way sufficiently far into it. In that respect, it is like music, drama, painting,  philosophy, natural sciences, and many other intellectual human activities. This is a perfectly valid reason to provide everyone with an opportunity to sample it, and make it possible for those who like what they see to pursue it as far as they desire. What it is not, is a valid reason for making learning math obligatory throughout elementary, middle, and high school education.

The argument behind math’s obligatory status in education is that it is useful; more precisely, it is useful in the practical, everyday world. This is the view of mathematics I am adopting in the short series of “Devlin’s Angle” essays of which this is the third. (There will be one more next month. See episode 1 here and episode 2 here.)

Indeed, mathematics is useful in the everyday practical world. In fact, we live in an age where mathematics is more relevant to our lives than at any previous time in human history. 

It is, then, perfectly valid to say that we force each generation of school students to learn math because it is a useful skill in today’s world. True, there are plenty of people who do just fine without having that skill, but they can do so only because there are enough other people around who do have it.

But let’s take that argument a step further. How do you teach mathematics so that it prepares young people to use it in the world? Clearly, you start by looking at the way people currently use math in the world, and figure out how best to get the next generation to that point. (Accepting that by the time those students finish school, the world’s demands may have moved on a bit, so those new graduates may have a bit of catch up and adjustment to make.)

If the way the professionals use math in the world changes, then the way we teach it should change as well.  Don’t you think? That’s certainly what has happened in the past.

For instance, in the ninth century, the Arabic-Persian speaking traders around Baghdad developed a new, and in many instances more efficient, way to do arithmetic calculations at scale, by using logical reasoning rather than arithmetic. Their new system, which quickly became known as al-jabr after one of the techniques they developed to solve equations, soon found its way into their math teaching.

When Hindu-Arabic arithmetic was introduced into Europe in the thirteenth century, the school systems fairly quickly adopted it into their arithmetic teaching as well. (It took a few decades, but knowledge moved no faster than the pace of a packhorse back then. I tell the story of that particular mathematics-led revolution in my 2011 book The Man of Numbers.)

The development of modern methods of accounting and the introduction of financial systems such as banks and insurance companies, which started in Italy around the same time, also led to new techniques being incorporated into the mathematical education of the next generation.

Later, when the sixteenth century French mathematician François Viète introduced symbolic algebra, it too became part of the educational canon.

In each case, those advances in mathematics were introduced to make mathematics more easy to use and to increase its application. There was never any question of “What is this good for?” People eagerly grabbed hold of each new development and made everyday use of it as soon as it became available.

The rise of modern science (starting with Galileo in the seventeenth century) and later the Industrial Revolution in the nineteenth century, led to still more impetus to develop new mathematical concepts and techniques, though some of those developments were geared more toward particular groups of professionals. (Calculus, for example.)

To make it possible for an average student or worker to make use of each new mathematical concept or technique, sets of formal calculating rules (algorithmic procedures) were developed and refined. Once mastered, these made it possible to make use of the new mathematics to handle—in a practical way—the tasks and problems of the everyday world for which those concepts and techniques had been developed to deal with in the first place.

As a result of all those advances, by the time the Baby Boomers came onto the educational scene in the 1950s, the curriculum of mathematical algorithms that were genuinely important in everyday life was fairly large. It was no longer possible for a student to understand all the underlying mathematical concepts and techniques behind the algorithms and procedures they had to learn. The best that they could do was master, by repetitive practice, the algorithmic procedures as quickly as possible and move on. [A few of us had difficulty doing that. We wanted to understand what was going on. By and large, we frustrated our teachers, who seemed to think we were simply troublesome slow learners. Some of us eventually learned to “play the mindless algorithm game” in class to pass the test, but kept struggling on our own to understand what was going on, setting us on a path to becoming mathematics professors in the 1970s.]

It was while that Boomer generation was going through the school system that mathematics underwent the first step of a seismic shift that within a half of a century would completely revolutionize the way mathematics was done. Not the pure mathematics practiced by a few specialists as an art—though that too would be impacted by the revolution to some extent. Rather, it was mathematics-as-used-in-the-world that would be radically transformed.

The first step of that revolution was the introduction of the electronic desktop calculator in 1961. Although, mechanical desktop calculators had been available since the turn of the Twentieth Century, by and large their use was restricted to specialists—often called “computers” in businesses. [I actually had a summer-job with British Petroleum as such a specialist in my last three years at high school, and it was in my final year in that job that the office I worked in acquired its first electronic desktop calculator and the British Petroleum plant bought its first digital computer, both of which I learned to use.] But with the increasing availability of electronic calculators, and in particular the introduction of pocket-sized versions in the early 1970s, their use in the workplace rapidly became ubiquitous. Mathematics underwent a major change. Humans no longer needed to do arithmetic calculations themselves, and professionals using arithmetic in their work no longer did.

It was not too many years later that, one by one, electronic systems were developed that could execute more and more mathematical procedures and techniques, until, by the late 1980s, there were systems that could handle all the mathematical procedures that constituted the bulk of not only the school mathematics curriculum, but the entire undergraduate math curriculum as well. The final nail in the coffin of humans needing to execute mathematical procedures was the release of the mathematics system Mathematica in 1988, followed soon after by the release of Maple.

In the scientific, industrial, engineering, and commercial worlds, each new tool was adopted as soon as it became available, and since the early 1990s, professionals using mathematical techniques to carry out real-world tasks and solve real-world problems have done so using tools like Mathematica, Maple, and a host of others that have been developed.

Simultaneously, colleges and universities quickly incorporated the use of those new tools into their teaching. And while the cost of the more extensive tools put their use beyond most schools, the graphing calculator too was quickly brought into the upper grades of the K-12 system, after its introduction in 1990.

Yet, while the pros in the various workplaces changed over to the new human-machine-symbiotic way of doing math with little hesitation, most educators, exhibiting very wise instincts, proceeded with far more caution. The first wave of humans to adopt the new, machine-aided approach had all learned mathematics in an age when you had to do everything yourself. Back then, “computers” were people. For them, it was easy and safe to switch to executing a few keystrokes to make a computer run a procedure they had carried out by hand many times themselves. But how does a young person growing up in this new, digital-tools-world learn how to use those new tools safely and effectively?

To some extent, the answer is (and was) obvious. You teach not for smooth, proficient, accurate execution of procedures, but for broad, general understanding of the underlying mathematics. The downplay of execution and increased emphasis on understanding are crucial. Computers outperform us to ridiculous degrees (of speed, accuracy, size of dataset,  and information storage and retrieval) when it comes to execution of an algorithm. But they do not understand mathematics. They do not understand the problem you are working on. They do not understand the world. They don't understand anything. 

People, on the other hand, can understand, and have a genetically inherited desire to do so.

But just how do you go about teaching for the kind of understanding and mastery that is required for students to transition into worlds and workplaces dominated by a wide array of new mathematical tools, where they will encounter work practices that involve very little by way of hand execution of algorithms?

We know so little about how people learn (though we do know a whole lot more than we did just a few decades ago), that most of us with a stake in the education business are rightly concerned about making any change that would effectively be a massive experiment on an entire generation. So we can, and should, expect small steps, particularly in systemic education.

In the U.S., the mathematicians who developed the mathematical guidelines for the Common Core State Standards made a good first attempt at such a small step. True, it quickly ran into difficulties when it came to implementing the guidelines in a large and complex public educational system that is answerable to the public. But that is surely a temporary hiccup. Most of the problems at launch came from a lack of effective ways to assess the new kind of learning. Those problems can be and are being fixed. Which is just as well. For, although it’s possible to argue for tinkering with specific details of the Common Core State Standards guidelines, in terms of setting out a broad set of educational goals to aim for, there is no viable alternative first step. The pre-1970s educational approach is no longer an option.

In the meantime, individual teachers at some schools (particularly, but not exclusively, private schools) have been trying different approaches, in some cases sharing their experiences on the MTBOS (Math Twitter Blog-O-Sphere), making use of another technological tool (social media) now widely available. [For a quick overview of one global initiative to support and promote such innovations, the OECD’s Innovative Pedagogies for Powerful Learning project (IPPL), see this recent article from the Brookings Institution.]

The mini-course I gave at Nueva School in the San Francisco Bay Area last January, which I talked about in the first of this short series of essays, is one such experiment in teaching mathematics in a way that best prepares the next generation for the world they will live and work in after graduation. I tested it first with a class of non-science majors in Princeton in the fall of 2015 and then again with an Adult Education class at Stanford in the fall of 2017. The Nueva School class was its third outing.

With the above backstory now established, next month I will describe that course and talk about how today’s pros “do the math”. (Again, let me stress, I am not talking here about “pure math”, the academic discipline carried out by professional mathematicians in universities and a few think tanks. My focus here is on using math in the everyday world.)

In the meantime, I’ll leave you with a simple arithmetic problem that I will discuss in detail next time.

It comes with two instructions:

  1. Solve it as quickly as you can, in your head if possible. Let your mind jump to the answer.
  2. Then, and only then, reflect on your answer, and how you got it.
The goal here is not to get the right answer, though a great many of you will. Rather, the issue is how do our minds work, and how can we make our thinking more effective in a world where machines execute all the mathematical procedures for us?

Ready for the problem? Here it is. 

PROBLEM: A bat and a ball cost $1.10. The bat costs $1 more than the ball. How much does the ball cost on its own? (There is no special pricing deal.)

Wednesday, February 7, 2018

How today’s pros solve math problems: Part 1

Last month, I wrote about my recent experience teaching a three-day mini-course in the Nueva School January electives “Intersession” program. What I left out was a description of the course itself. I ended with the below diagram as a teaser. I said that, when reading in the usual left-right-down reading order, these were the technology tools that I typically turn to when I start to work on solving a new problem.


A number of mathematicians commented on social media that their list would be almost identical to mine. That did not surprise me. My chart simply captures the way today’s pros approach new problems. A number of math teachers expressed puzzlement. That too did not surprise me. The current mathematics curriculum is still rooted in a conception of “doing math” that developed to meet society’s needs in the 19th Century.

Actually, I should point out that the diagram above is not exactly the one I published last month. I have added an icon for a spreadsheet. A mathematician in Austria emailed me to say I should have included it. The two of us had corresponded in the past about the use of spreadsheets in mathematics, both in problem solving and in teaching, and we were (and are) very much on the same page as to their usefulness in a wide variety of circumstances. My excuse for overlooking it the first time round was that it was only the second technological tool I brought into my mathematics arsenal, so far back in my career that I had long ago stopped thinking of it as something new. (The first piece of “new tech” I adopted was the electronic calculator, and that too did not appear in my chart.) I suspect that almost all math teachers, and indeed, pretty well all of society, make frequent use of calculators and spreadsheets, not only in their professional activities but in their social and personal lives as well. Still, the spreadsheet is such a powerful, ubiquitous mathematics tool, I should have included it, and now I have. (Its use definitely figured in the guidance I gave to the Nueva School class.) I have placed it in the position in my list that, on reflection, I find I turn to in order of frequency.

Some of the responses I received from teachers indicated that I need to clarify that, by “solving a mathematical problem”, I mean using mathematics to solve a real-world problem. The problem we worked on at Nueva School was one UPS worked on not long ago: “What is the most efficient way to route packages from place to place?” More on that later. A simpler example in the same vein is when we ask ourselves “Which kind, model, and hardware configuration of mobile phone best meets my needs within my current budget?”—an example where, for most of us, the item’s cost is high enough for us to weigh the (many) options fairly carefully.

This is clearly not the same as “solving a math problem” in a typical math textbook. For example, “What are the roots of the equation x2 + 3x – 5 = 0?” Those kinds of questions are, of course, designed to provide practice in using various specific, sharply focused, mathematics techniques, procedures, formulas, or algorithms.

Those techniques, procedures, etc. are the basic building blocks for using mathematics to solve problems in real life, but they don’t really present much of a problem, in the sense the word is used outside the math class. Indeed, the reason it can be valuable to master those basic techniques, etc. is that being able to use them fluidly means they won’t be a problem (in the sense of an obstacle) that gets in the way of solving what really is a mathematical problem (e.g., which phone to buy). That, of course, is why we call them basic skills. But having mastery of a range of basic skills does not make a person a good problem solver any more than being a master bricklayer makes someone an architect or a construction engineer.

My focus then, is on using math to solve real-world problems. That’s where things are very different from the days when I first learned mathematics. Back in the 1950s and 60s, when I went through the school system, we spent a huge amount of time mastering algorithms and techniques for performing a variety of different kinds of numerical and symbolic calculations, geometric reasoning, and equation solving. We had to. In order to solve any real-world problem, we had to be able to crank the algorithmic and procedural handles.

Not any more. These days, that smartphone in your pocket has access to cost-free cloud resources that can execute any mathematical procedure you need. What’s more, it will do it with a dataset of any real-world size, with total accuracy to whatever degree you demand, and in the majority of cases in a fraction of a second.

To put it another way, all those algorithms, techniques, and procedures I spent years mastering, all the way through to earning my bachelors degree in mathematics, became obsolete within my lifetime, an observation I wrote about in an article in the Huffington Post in January of last year.

So, does that mean all that effort was wasted? Not at all. Discounting the fact that in my case, I was able to make good use of those skills and knowledge for several decades before the march of technology rendered them obsolete, the one thing that I gained as a result of all that procedural learning that is as valuable today as it was back then, was the ability to think mathematically. I wrote about one aspect of that “mathematical thinking” mental ability, number sense, in a simultaneously published follow-up piece to that Huffington Post article.

In today’s world, all the algorithmic, computational, algebraic, geometric, logical, and procedural skills that used to take ten years of effort to master can now be bought for $699. At least, that amount (the price of an iPhone 8, which I chose for illustration) is all it costs to give you access to all those skills. Making effective use of that vast powerhouse of factual knowledge and procedural capacity requires considerable ability. Anyone who mastered mathematics the way I did acquired that ability as an automatic by-product of mastering the basic skills. But what does it take to acquire it in an age when all those new tools are widely available?

The answer, of course (though not everyone involved in the mathematics education system thinks it is obvious, or even true), is that the educational focus has to shift from procedural mastery to understanding. Which is precisely the observation that guided the Common Core initiative in the United States. Yes, I know that the current leadership of the US Department of Education believes that the Common Core is a bad idea, but that is an administration that also believes the future of energy lies in fossil fuels, not renewables, and the highly qualified, career-professional contacts I have in the Department of Education have a very different view.

How do you acquire that high-level skill set? The answer is, the same way people always did: through lots of practice.

But be careful how you interpret that observation. What need to be practiced are the kinds of activities that you would use as a professional—or at least a competent user of mathematics—in the circumstances of the day. In my school days, that meant we had to practice with highly constrained, “toy” problems. But with today’s technologies, we can practice on real-world problems using real-world data.

Almost inevitably, when you do that, you find you frequently need to drop down to suitably chosen “toy problem” variants of your task in order to understand how a particular online tool (say) works and what it can (and cannot) do. But today, the purpose of, say, inverting a few 2x2 or 3x3 matrices is not (as it was in my day) so you can become fluent at doing so, and certainly not because you will actually invert by hand that 100x100 matrix that has just reared its ugly head in your real-world problem. No, you just need to get a good understanding of what it means to invert a matrix, why you might do so, and what problems can arise.

And you know what? That’s rarely a problem. Once you have identified a mathematical technique you need to understand, the chances are high you will find not one but a dozen or more YouTube videos that explain it to you.

These new tools certainly don’t solve the problem for you. [Well, sometimes they may do, but in that case it wasn’t a problem that required the time of a mathematician. Better to move on and put your efforts into a problem that cannot be solved by an app in the Cloud!] All that these fancy new tools have done is change the level at which we humans operate.

At heart, that shift is no different from the level-shift introduced in the 9th Century when traders in and around Baghdad developed techniques for doing routine arithmetic calculations at scale, by performing operations not on specific numbers but on classes of numbers. One of the techniques they developed was called, al-jabr, a term that ended up giving the name we use today to refer to that new kind of calculation procedure: algebra.

Throughout mathematics’ history, mathematicians have calculated and reasoned logically with the basic building blocks of the time. Today’s procedures (that have to be executed) turn into tomorrow’s basic entities (on which you operate). A classic example is differential calculus, where functions are no longer viewed as rules that you execute to yield new numbers from old numbers, but higher-level objects on which you operate to produce new functions from old functions.

So (finally), what exactly did we do in that Nueva School mini-course to illustrate the way today’s pros use math to solve a problem? The problem, remember, was this: Reverse engineer the core algorithm than UPS uses to route packages from origin to destination?

To start the class off—they worked in small teams of three or four—I provided a small amount of information to get them started:
  1. Tracking information for a fairly large, heavy case, including a partially dismantled bicycle, I had shipped from Petaluma, California to Fair Haven, New Jersey, in 2015. See image below.
  2. I told them I sent the case by “three day select.”
  3. I reported that my package went by plane from Louisville, Kentucky, to the UPS facility in Newark, where it was immediately loaded onto a truck, and was delivered to the intended Fair Haven destination with just a few hours to spare within the three-day period guaranteed.
That information, I told the class, was enough to figure out how the routing algorithm worked. [This itself is useful information that I did not have when I first solved the problem, but they had to figure it out by the end of the course, so I was happy to give them additional information.] In solving this problem, they could elicit my help as their “math consultant,” to call on with specific questions when required. But they had to carry out the key steps.

They could, of course, use the various tools in my “modern math tools” chart, and any others they could find. (Since the UPS routing algorithm is an extremely valuable trade secret, they would not find that online, of course.)

Next month, I’ll tell you how they got on. In the meantime, you might like to see how far you can get with it. Happy problem solving! Happy mathematical thinking!

Part 2 will appear next month.

Tuesday, January 23, 2018

Déjà vu, all over again

I gave a short course at a local high school recently. Three days in a row, two hours a day, to fifteen students. To my mind, it was a huge success. By the end of the course, the students had successfully reverse-engineered UPS’s core routing/scheduling algorithm. In fact, they spent the last half hour brainstorming how UPS might improve their efficiency. (My guess is the company had long ago implemented, or at least considered, the ideas the kids came up with, but that simply serves to illustrate how far they had come in just six hours of class-time.)

To be sure, it was not an average class in an average high school. Nueva School, located in the northern reaches of Silicon Valley, is private and expensive (tuition runs at $36,750 for an 8th gader), and caters to students who have already shown themselves to be high achievers. Many Silicon Valley tech luminaries send their children there, and some serve on the board. They have an excellent faculty. Moreover, the fifteen students in my class had elected to be there, as part of their rich, January, electives learning experience called “Intersession”.

I was familiar with the school, having been invited to speak at their annual education conference on a couple of occasions, but this was the first time I had taught a class.

Surprisingly, the experience reminded me of my own high school education, back in the UK in the early 1960s. My high school was a state run, selective school in the working class city of Hull, a major industrial city and large ocean fishing and shipping port. Socially and financially, it was about as far away as you could get from Nueva School on the San Francisco Peninsula, and my fellow students came from very different backgrounds than the students at Nueva.

What made my education so good was a highly unusual set of historical circumstances. Back then, Hull was a fiercely socialist city that, along with the rest of the UK, was clawing its way out of the ravages of the Second World War. For a few short years, the crippling English class system broke down, and an entire generation of baby boomers entered the school system determined to make better lives for themselves—and everyone else. (“Me first” came a generation later.)

We had teachers who had either just returned from fighting the war (the men on the battlefields, the women in the factories or in military support jobs), or were young men and women just starting out on their teaching careers, having received their own school education while the nation was at war. There was a newly established, free National Health Service, an emerging new broadcasting technology (television) run by a public entity, a rapidly growing communications systems (a publicly funded telephone service), and free education, including government-paid- for university education for the 3 percent or so able to pass the challenging entrance exams.

We were the generation that the nation was dependent on to rebuild, making our way through the education system in a social and political environment where the class divisions that had been a part of British life for centuries had been (temporarily, it turned out) cast aside by the need to fight a common enemy across the English Channel. The result was that, starting in the middle of the 1960s, a “British Explosion” of creative scientific, engineering, and artistic talent burst forth onto the world. Within our individual chosen domains, we all felt we could do anything we set our minds to. And a great many of us did just that. About half my high school class became highly successful people. That from a financially impoverished, working class background.

It was short lived, lasting but a single generation. I was simply lucky to be part of it.

What brought it all back to me was finding myself in a very similar educational environment in my three days at Nueva School. The circumstances could hardly be more different, of course. But talking and working with those students, I sensed the same thirst to learn, the same drive to succeed (in terms they set for themselves), and the same readiness to keep trying I had experienced two generations earlier. It felt comfortingly—and encouragingly—familiar.

But I digress. In fact, I’ve done more than digress. I’ve wandered far from my intended path. Or have I? The point I want to get across is that when it comes to learning, success is about 5 percent talent, 35 percent the teachers and students around you, and 60 percent desire and commitment. (I just made up those figures, but they represent more or less how I see the landscape, having been an education professional for half a century.)

It turns out that, in today’s world, given those ingredients, in roughly those proportions, it is possible for a small group of people, in the space of just a few days, to make significant progress in solving a major problem of massive societal importance. (If you can figure out how UPS performs its magic, you can do the same thing with many other large organizations, Walmart, Amazon, United Airlines, and so on.)

How can it be possible to take a small group of students, still in high school, and make solid progress on a major mathematical problem like that? It would not have been possible in my school days. The answer is, in today’s world, everyone has access to the same rich toolset the professionals use. Moreover, most of those tools—or at least, enough of them—are free to anyone with access to a smartphone or a personal computer. You just have to know how to make effective use of them.

Next month, I will describe how my Nueva class went about the UPS project. (I had done it once before, with a non-science majors undergraduate class at Princeton University. Doing it with high school students confirmed my belief that a group with less academic background could achieve the same result, in the process providing me with some major-league ammunition to back up my oft-repeated—and oft-ignored or disputed—claim that K-12 mathematics education is in need of a major (and I mean MAJOR) makeover. (After the invention of the automobile, it made more sense to teach people how to drive than how to look after a horse. I feel the math ed argument should end with that razor-sharp analogy, but it rarely does.)

As I say, that discussion is for next month. But let me leave you with a teaser. Actually, two teasers. One is my January 1, 2017 opinion piece in the Huffington Post, "All The MathematicalMethods I Learned In My University Math Degree Became Obsolete In My Lifetime." The other teaser is the diagram I will end with. It summarizes some of the most useful tools that a professional mathematician today uses when starting to work on a new problem. (Note: I’m talking about using math to solve real-world problems here. Pure mathematics is very different, although all the tools I will mention can be of use to a pure mathematician.)

This is my set of “most useful tools,” I should note, and reading the diagram left-to- right, top to bottom, the tools I list are roughly in the order I have used them in working on various projects over the past fifteen years. Other mathematicians might produce different collections and different orders. But they won’t be that much different, and I’ll bet they all begin with the same first tool.

If you find this diagram in any way surprising, you likely have not worked in today’s world of mathematical problem solving. If you find it surprising and are in mathematics education, I respectfully point out that this is the mathematical toolset that your students will need to master in order to make use of math in the world they will inhabit after graduation. You may or may not like that. If you don’t like it, then that is unfortunate. Mathematical problem solving is simply done differently today. It just is.





Thursday, December 14, 2017

Clash of representations


The pie chart in the above tweet jumped out of the page when it appeared in my twitter feed on September 14. My initial shock at seeing the figure 1% attached to a region of the pie chart that was evidently almost 25% of the total area of the disk did not last long, of course, since the accompanying text made it clear what the diagram was intended to convey. The 1% label referred to the section of the population being discussed, whereas the pie-chart indicated the share of taxes paid by that group. Indeed, the image was an animated GIF; when I clicked on it, the region labeled “1%” shrank, culminating with the chart on the right in the image shown below:
But here’s the thing. Even after I had figured out what the chart was intended to convey, I still found it confusing. I wondered if a lay-reader, someone who is not a professional mathematician, would manage to parse out the intended meaning. It was not long before I found out. The image below shows one of the tweets that appeared in response less than an hour later:
As I had suspected, a common reaction was to dismiss the chart as yet another example of a bad data visualization created by an innumerate graphics designer. Indeed, that had been my initial reaction. But this particular example is more interesting. Yes, it is a bad graphic, for the simple reason that it does not convey the intended message. But not because of the illustrator’s innumeracy. In fact, numerically, it appears to be as accurate as you can get with a pie chart. The before and after charts do seem to have regions whose areas correspond to the actual data on the tax-payer population.

This example was too good to pass up as an educational tool: asking a class to discuss what the chart is intended to show, could lead to a number of good insights into how mathematics can help us understand the world, while at the same time having the potential to mislead. I was tempted to write about it in my October post, but wondered if I should delay a couple of months to avoid an example that was at the heart of a current, somewhat acrimonious party-political debate. As it turned out, the September 30 death of the game-show host Monty Hall resolved the issue for me—I had to write about that—and then November presented another “must do” story (the use of mathematics in election jerrymandering). So this month, with the background political, tax votes now a matter of historical record, I have my first real opportunity to run this story.

The two-month delay brought home to me just how problematic this particular graphic is. Even knowing in advance what the issue is, I still found I had to concentrate to “see” the chart as conveying the message intended. That “1%” label continued to clash with the relative area of the labeled region.

It’s a bit like those college psychology-class graphics that show two digits in different font sizes, and ask you to point to the digit that represents the bigger integer. If the font sizes clash with the sizes of the integers, you take measurably longer to identify the correct one, as shown below:
For me, the really big take-home lesson from the tax-proposal graphic is the power of two different mathematical representations of proportions: pie charts and numerical percentages. Each, on its own, is instant. In the case of the pie chart, the representation draws on the innate human cognitive ability to judge relative areas in simple, highly symmetrical figures like circular disks or rectangles. With percentages, there is some initial learning required—you have to understand percentages—but once you have done that, you know instantly what is meant by figures such as “5%” or “75%."

But how do you get that understanding of the meaning of numerical percentages? For most of us (I suspect all of us), it comes from being presented (as children) with area examples like pie charts and subdivided rectangles. This sets us up to be confused, bigly, by examples where those two representations are used in the same graphic but with the percentage representing something other than the area of the segment (or what that area is intended to represent).

The message then, from this particular example—or at least the message I got from it—is that powerful graphics are like any powerful tool, their power for good depends on using them wisely; if used incorrectly, they can confuse and mislead. And make no mistake about it, numbers are incredibly powerful tools. Their invention alone is by far the greatest mathematical invention in human history. That’s why in every nation in the world, math is the only mandated school subject apart from the native language.

Thursday, November 16, 2017

Mathematics and the Supreme Court

American courts have never appeared to be very receptive to mathematical arguments, in large part, some (including me) have assumed, because many judges do not feel confident evaluating mathematical reasoning and, in the case of jury trials, no doubt because they worry that unscrupulous, math-savvy lawyers could use formulas and statistics to fool jury members. There certainly have been some egregious examples of this, particularly when bogus probability arguments have been presented. Indeed, one classic misuse of conditional probability is now known as the “prosecutor’s fallacy."

Another example where the courts have trouble with probability is in cases involving DNA profiling, particularly Cold Hit cases, where a DNA profile match is the only hard evidence against a suspect. I myself have been asked to provide expert testimony in some such cases, and I wrote about the issue in this column in September and October of 2006.

In both kinds of case, the courts have good reason to proceed with caution. The prosecutor’s fallacy is an easy one to fall into, and with Cold Hit DNA identification there is a real conflict between frequentist and Bayesian probability calculations. In neither case, however, should the courts try to avoid the issue. When evidence is presented, the court needs to have as accurate an assessment as possible as to its reliability or veracity. That frequently has to be in the form of a probability estimate.

Now the courts are facing another mathematical conundrum. And this time, the case has landed before the US Supreme Court. It is a case that reaches down to the very foundation of our democratic system: How we conduct our elections. Not how we use vote counts to determine winners, although that is also mathematically contentious, as I wrote about in this column in November of 2000, just before the Bush v Gore Presidential Election outcome ended up before the Supreme Court. Rather, the issue before the Court this time is how states are divided up into electoral districts for state elections.

How a state carves up voters into state electoral districts can have a huge impact on the outcome. In six states, Alaska, Arizona, California, Idaho, Montana, and Washington, the apportioning is done by independent redistricting commissions. This is generally regarded—at least by those who have studied the issue—as the least problematic approach. In seven other states, Arkansas, Colorado, Hawaii, Missouri, New Jersey, Ohio, and Pennsylvania, politician commissions draw state legislative district maps. In the remaining 37 states, the state legislatures themselves are responsible for state legislative redistricting. And that is where the current problem arises.

There is, of course, a powerful temptation for the party in power to redraw the electoral district maps to favor their candidates in the next election. And indeed, in the states where the legislatures draw the maps, both major political parties have engaged in that practice. One of the first times this occurred was in 1812, when Massachusetts governor Elbridge Gerry redrew district boundaries to help his party in an upcoming senate election. A journalist at the Boston Gazette observed that one of the contrived districts in Gerry’s new map looked like a giant salamander, and gave such partisan redistricting a name, combining Gerry and mander to create the new word gerrymander. Though Gerry lost his job over his sleight-of- hand, his redistricting did enable his party to take over the state senate. And the name stuck.

Illegality of partisan gerrymandering is generally taken to stem from the 14th Amendment, since it deprives the smaller party of the equal protection of the laws, but it has also been argued to be, in addition, a 1st Amendment issue—namely an apportionment that has the purpose and effect of burdening a group of voters’ representational rights.

In 1986, the Supreme Court issued a ruling that partisan gerrymandering, if extreme enough, is unconstitutional, but it has yet to throw out a single redistricting map. In large part, the Supreme Court’s inclination to stay out of the redistricting issue is based on a recognition that both parties do it, and over time, any injustices cancel out, as least numerically. Historically, this was, generally speaking, true. Attempts to gerrymander have tended to favor both parties to roughly the same extent. But in 2012, things took a dramatic turn with a re-districting process carried out in Wisconsin.

That year, the recently elected Republican state legislature released a re-districting map generated using a sophisticated mathematical algorithm running on a powerful computer. And that map was in an altogether new category. It effectively guaranteed Republican majorities for the foreseeable future. The Democrat opposition cried foul, a Federal District Court agreed with them, and a few months ago the case found its way to the Supreme Court.

That the Republicans come across as the bad actors in this particular case is likely just an accident of timing; they happened to come to power at the very time when political parties were becoming aware of what could be done with sophisticated algorithms. If history is any guide, either one of the two main parties would have tried to exploit the latest technology sooner or later. In any event, with mathematics at the heart of the new gerrymandering technique, the only way to counter it may be with the aid of equally sophisticated math.

The most common technique used to gerrymander a district is called “packing and cracking." In packing, you cram as many of the opposing party’s voters as possible into a small number of “their” districts where they will win with many more votes than necessary. In cracking, you spread opposing party’s voters across as many of “your” districts as possible so there are not enough votes in any one of those districts to ever win there.

A form of packing and cracking arises naturally when better-educated liberal-leaning voters move into in cities and form a majority, leaving those in rural areas outnumbered by less-educated, more conservative-leaning voters. (This is thought to be one of the factors that has led to the increasing polarization in American politics.) Solving that problem is, of course, a political one for society as a whole, though mathematics can be of assistance by helping to provide good statistical data. Not so with partisan gerrymandering, where mathematics has now created a problem that had not arisen before, for which mathematics may of necessity be part of the solution.

When Republicans won control of Wisconsin in 2010, they used a sophisticated computer algorithm to draw a redistricting map that on the surface appeared fair—no salamander-shaped districts—but in fact was guaranteed to yield a Republican majority even if voter preferences shifted significantly. Under the new map, in the 2012 election, Republican candidates won 48 percent of the vote, but 60 of the state’s 99 legislative seats. The Democrats’ 51 percent that year translated into only 39 seats. Two years later, when the Republicans won the same share of the vote, they ended up with 63 seats—a 24-seat differential.

Recognizing what they saw as a misuse of mathematics to undermine the basic principles of American democracy, a number of mathematicians around the country were motivated to look for ways to rectify the situation. There are really two issues to be addressed. One is to draw fair maps—a kind of “positive gerrymandering.” The other is to provide reliable evidence to show that a particular map has been intentionally drawn to favor one party over another, if such occurs, and moreover to do so in a way that the courts can understand and accept. Neither issue is easy to solve, and without mathematics, both are almost certainly impossible.

For the first issue, a 2016 Supreme Court ruling gave a hint about what kind of fairness measure it might look kindly on: one that captures the notion of “partisan symmetry,” where each party has an equal opportunity to convert its votes into seats. The Wisconsin case now presents the Supreme Court with the second issue.

When, last year, a Federal District Court in Wisconsin threw out the new districting map, they cited both the 1st and 14th Amendments. It was beyond doubt, the court held, that the new maps were “designed to make it more difficult for Democrats, compared to Republicans, to translate their votes into seats.” The court rejected the Republican lawmakers’ claim that the discrepancy between vote share and legislative seats was due simply to political geography. The Republicans had argued that Democratic voters are concentrated in urban areas, so their votes have an impact on fewer races, while Republicans are spread out across the state. But, while that is true, geography alone does not explain why the Wisconsin maps are so skewed.

So, how do you tell if a district is gerrymandered? One way, that has been around for some time, is to look at the geographical profile. The gerrymandering score, G, is defined by:
G = gP/A, where
g: the district’s boundary length, minus natural boundaries (like coastlines and rivers)
P: the district’s total perimeter
A: the district’s area
The higher the score, the wilder is the apportionment as a geographic region, and hence the more likely to have been gerrymandered.

That approach is sufficiently simple and sensible to be acceptable to both society and the courts, but unfortunately does not achieve the desired aim of fairness. And, more to the point in the Wisconsin case, use of sophisticated computer algorithms can draw maps that have a low gerrymandering score and yet are wildly partisan.

The Wisconsin Republicans’ algorithm searched through thousands of possible maps looking for one that would look reasonable according to existing criteria, but would favor Republicans no matter what the election day voting profile might look like. As such, it would be a statistical outlier. To find evidence to counter that kind of approach, you have to look at the results the districting produces when different voting profiles are fed into it.

One promising way to identify gerrymandering is with a simple mathematical formula suggested in 2015, called the “efficiency gap." It was the use of this measure that caused, at least in part, the Wisconsin map to be struck down by the court. It is a simple idea—and as I noted, simplicity is an important criterion, if it is to stand a chance of being accepted by society and the courts.

You can think of a single elector’s vote as being “wasted” if it is cast in a district where their candidate loses or it is cast in a district where their candidate would have won there anyway. The efficiency gap measures those “wasted” votes. For each district, you total up the number of votes the winning candidate receives in excess of what it would have taken to elect them in that district, and you total up the number of votes the losing candidate receives. Those are the two parties’ “wasted votes” for that district.

You then calculate the difference between those “wasted-vote” totals for each of the two parties, and divide the answer by the total number of votes in the state. This yields a single percentage figure: the efficiency gap. If that works out to be greater than 7%, the systems developers suggest, the districting is unfair.

By way of an example, let’s see what the efficiency gap will tell us about the last Congressional election. In particular, consider Maryland’s 6 th Congressional district, which was won by the Democrats. It requires 159K votes to win. In the last election, there were 186K Democrat votes, so 186K – 159K = 26K Democrat votes were “wasted,” and 133K Republican votes, all of which were “wasted.”

In Maryland as a whole, there were 510K Democrat votes “wasted” and 789K Republican votes “wasted.” So, statewide, there was a net “waste” of 789K – 510K = 279K Republican votes.

There were 2,598M votes cast in total. So the efficiency gap is 279K/2598K = 10.7% in favor of the Democrats.

I should note, however, that the gerrymandering problem is currently viewed as far more of a concern in state elections than in congressional races. Last year, two social scientists published the results they obtained using computer simulations to measure the extent of intentional gerrymandering in congressional district maps across most of the 50 states. They found that on the national level, it mostly canceled out between the parties. So banning only intentional gerrymandering would likely have little effect on the partisan balance of the U.S. House of Representatives. The efficiency gap did, however, play a significant role in the Wisconsin court’s decision.

Another approach, developed by a team at Duke University, takes aim at the main idea behind the Wisconsin redistricting algorithm—searching through many thousands of possible maps looking for ones that met various goals set by the creators, any one of which would, of necessity, be a statistical outlier. To identify a map that has been obtained in this way, you subject it to many thousands of random tweaks. If the map is indeed an outlier, the vast majority of tweaks will yield a fairly unremarkable map. So, you compare the actual map with all those thousands of seemingly almost identical, and apparently reasonable, variations you have generated from it. If the actual map produces significantly different election results from all the others, when presented with a range of different statewide voting profiles, you can conclude that it is indeed an “outlier” — a map that could only have been chosen to deliberately subvert the democratic process.

And this is where we—and the Supreme Court—are now. We have a problem for our democracy created using mathematics. Mathematicians looking for mathematical ways to solve it, and there are already two candidate “partisan gerrymandering test” in the arena. Historically, the Supreme Court has proven resistant to allowing math into the courtroom. But this time, it looks like they may have no choice. At least as long as state legislatures continue to draw the districting maps. Maybe the very threat of having to deal with mathematical formulas and algorithms will persuade the Supreme Court to recommend that Congress legislates to enforce all states to use independent commissions to draw the districting maps. Legislation under pain of math. We will know soon enough.