Tuesday, September 28, 2010

Women make less money than men on average, but how much (if any) of this is due to sexism/discrimination?

Last week Christina Hoff Sommers had a New York Times op-ed about the gap in men's and women's pay. The headline: "Women Don't Need the Paycheck Fairness Act." Sommers writes:

AMONG the top items left on the Senate’s to-do list before the November elections is a “paycheck fairness” bill, which would make it easier for women to file class-action, punitive-damages suits against employers they accuse of sex-based pay discrimination.

The bill’s passage is hardly certain, but it has received strong support from women’s rights groups, professional organizations and even President Obama, who has called it “a common-sense bill.”

But the bill isn’t as commonsensical as it might seem. It overlooks mountains of research showing that discrimination plays little role in pay disparities between men and women, and it threatens to impose onerous requirements on employers to correct gaps over which they have little control. . . .

[F]or proof, proponents point out that for every dollar men earn, women earn just 77 cents.

But that wage gap isn’t necessarily the result of discrimination. On the contrary, there are lots of other reasons men might earn more than women, including differences in education, experience and job tenure.

When these factors are taken into account the gap narrows considerably — in some studies, to the point of vanishing. A recent survey found that young, childless, single urban women earn 8 percent more than their male counterparts, mostly because more of them earn college degrees.

Moreover, a 2009 analysis of wage-gap studies commissioned by the Labor Department evaluated more than 50 peer-reviewed papers and concluded that the aggregate wage gap “may be almost entirely the result of the individual choices being made by both male and female workers.”
I agree with all of that. I don't have much of an opinion on the bill, since I haven't studied the provisions. I just want to focus on the underlying premise: that there's a significant discrimination-based gap in how much men and women are paid.

Over the weekend, the New York Times ran several letters rebutting the op-ed. If you know how these discussions tend to go, and if you're familiar with the NYT's letters section, you might be able to guess what the top letter says. Linda D. Hallman of the American Association of University Women (AAUW) writes:
The wage gap is real. Our 2007 report, “Behind the Pay Gap,” which controlled for factors flagged by Ms. Sommers, like education and experience, found that college-educated women earn less than men with comparable backgrounds.

The latest analysis Ms. Sommers cites, which shows young women outearning young men, needs to be viewed with a skeptical eye. The average American woman still earns 23 percent less than her male counterpart earns, a gap that is widest among older women and smallest among younger women.
Now, let's break down the main talking points from that letter:

1. The AAUW did a study that took into account Sommers's points, and they found that the gender gap is still "real" — women earn "less" than men.

2. "The average American woman still earns 23 percent less than her male counterpart earns."

3. The gap is "widest among older women and smallest among younger women."

Point 3 indicates that the gap is shrinking over time. It's easy to imagine that this trend would continue and eventually there'd be little or no gap, even without controlling for other factors.

How about points 1 and 2? If you read those in quick succession, you might go away with the impression that there's been a rigorous study that controlled for all the variables and still found a 23% pay gap between American male and female workers.

But that's not what Hallman says in her letter. She says the AAUW controlled for variables and found a gap . . . of unmentioned size. She says these are the same "factors flagged by Ms. Sommers" — implying that their report should allay the concerns Sommers expressed in her op-ed. Shortly after making these statements, she says there's a 23% gap.

But that 23% gap is before controlling for any variables. So that statistic is simply repeating the shortcoming that Sommers called out in her op-ed.

I wanted to see if that study Hallman links to did a better job of clarifying how much of the gap is actually due to gender itself, rather than other factors that happen to be correlated with gender. The link goes to an "Executive Summary" and a "Full Report" (which are both PDFs).

I don't see anything in the summary about controlling for variables. It simply reports the uncontrolled figures as if they're the definitive word on the "real" gap. We're supposed to see these statistics and immediately perceive sexism in how much employers pay their employees. But the gap alone doesn't demonstrate there's any sexism at play — it could result (in whole or in part) from benign factors that are correlated with gender.

So, how about the full report? I haven't read the whole thing — it's 45 pages, not counting the end materials. But they clearly found a lot of explanations for why there is such a gap, many of which they attribute to men's and women's different choices.

Here's one example of a factor, which I've taken almost at random: the report tells us that among full-time workers, men work longer hours than women (45 and 42 hours a week, respectively). This is also true among part-time workers (22 and 20 hours a week worked by men and women, respectively). Only 9% of female full-time workers work over 50 hours a week, compared with 15% of male full-time workers who work such long hours. (This is from page 15, and there's a relevant graph — only about full-time workers — on page 17.)

A little later, the AAUW gives us this conclusion, in a green, bold-faced heading:
A large portion of the gender pay gap is not explain by women's choices of characteristics.
Under that heading, the AAUW claims to support the conclusion:
If a woman and a man make the same choices, will they receive the same pay? The answer is no. The evidence shows that even when the "explanations" for the pay gap are included in a regression, they cannot fully explain the pay disparity. The regressions for earnings one year after college indicate that when all variables are included, about one-quarter of the pay gap is attributable to gender. That is, after controlling for all the factors known to affect earnings, college-educated women earn about 5 percent less than college-educated men earn. Thus, while discrimination cannot be measured directly, it is reasonable to assume that this pay gap is the product of gender discrimination.
Well, 5% is much smaller than the gap invoked in the NYT letter: 23%.

Now, you could sensibly respond: "But even a 5% difference in pay based on gender is unacceptable." Of course it would be unacceptable is women were paid 5% less than men due to their gender.

However,  even this controlling-for-variables statistic does not give us grounds to conclude that if you're a woman, you get paid 5% less than you would have if only you had been born male.

After all, how could the report have reached such a definitive conclusion? It firmly says "The answer [to whether a man and women who make the same choices will make the same money] is no," and this is proven by "the evidence." That presupposes that the AAUW in fact looked at all the relevant evidence. But the best anyone can do when they're studying such an immensely complex societal question is to control for some variables. It's an open question whether there are other relevant factors out there that the study ignored.

For instance, I said that the report looks at hours worked by full-time workers and hours worked by part-time workers. OK, that's nice. But there's some more information I'd like to know, which I don't see in the report's discussion of hours: how much more likely are men to work full-time rather than part-time? This question isn't answered by telling us how many more hours the full-time male workers work than their female counterparts. (As I said, I haven't read the whole report, so perhaps I'm wrong that the report fails to consider this factor. But you would think they'd mention it in the section about how many hours full-time and part-time workers work.) [UPDATE: Detailed discussion of this point in the comments. It's a little more complex than I thought when I was writing this post, but I still believe the report hasn't fully considered the distinction between full- and part-time workers.]

Now, what are all the variables the study failed to take into account? I don't know! And we're probably not going to find the answer to that question in the report; naturally, the report is going to talk about the things the researchers did study rather than talk about the factors they failed to study.

Thomas Sowell explains in his book Economic Facts and Fallacies (page 61):
Ideally, we would like to be able to compare those women and men who are truly comparable in education, skills, experience, continuity of employment, and full-time or part-time work, among other variables, and then determine whether employes hire, pay, and promote women the same as they do comparable men. At the very least, we might then see in whatever differences in hiring, pay and promotions might exist a measure of how much employer discrimination exists. Given the absence or imperfections of data on some of these variables, the most we can reasonably expect is some measure of whatever residual economic differences between women and men remain after taking into account those variables which can be measured with some degree of accuracy and reliability. That residual would then give us the upper limit of the combined effect of employer discrimination plus whatever unspecified or unmeasured variables might also exist.
In other words, the size of the gap attributable to gender discrimination might be 5%. Or it might be less than that. It might be 2% or 1%. It might be zero. It might even favor women. We don't know the answer.

If you're interested enough in this question to have read this far, I highly recommend buying Economic Facts and Fallacies and reading the chapter called "Male-Female Facts and Fallacies," where Sowell brilliantly explains many of the factors that account for the gender gap.

IN THE COMMENTS: LemmusLemmus draws an insightful analogy:
Sowell is right, of course. Just ascribing residual variance to your favourite factor, such as sexism, is the statistical equivalent of the God of the Gaps argument. [link added]

4 comments:

LemmusLemmus said...

"For instance, I said that the report looks at hours worked by full-time workers and hours worked by part-time workers. OK, that's nice. But there's some more information I'd like to know, which I don't see in the report's discussion of hours: how much more likely are men to work full-time rather than part-time?"

The regression contains a variable "hours worked per week" (p. 42).

"naturally, the report is going to talk about the things the researchers did study rather than talk about the factors they failed to study."

Actually, any serious social science paper on this kind of research would discuss limitations, but you can hardly expect that from what is, after all, a policy paper from an advocacy group.

As for uncontrolled factors, I note that the regression analysis does not control for whether or not the employees moved to a new place to secure the job they got. (How exactly you'd operationalize "new" in this context is a different question.) I don't know the literature on the question, but I would guess this is a relevant variable.

John Althouse Cohen said...

The regression contains a variable "hours worked per week" (p. 42).

That's a good point, and one I should have made clear in the post. I knew I was taking a risk by talking about the absence of something in the report when I hadn't read the whole report (as I did say in the post).

However, is controlling for "hours worked per week" sufficient to control for the percentage of women vs. men who worked full-time vs. part-time? Surely there's a very strong correlation between those two variables. But just looking at the average number of hours worked among all full- and part-time employees doesn't tell you the ratio of full to part time.

For instance, let's say men work an average of 38 hours per week, and women work an average of 35 hours per week (I'm just making this up, but it would be plausible based on the figures I cite in the post). That might give you a pretty good guess that women work part-time more often than men, but it still seems to be glossing over a significant fact about the full/part-time divide. There are other distinctions between part- and full-time jobs, such as how easy it is to get promoted (it's significantly harder for part-time workers). Also, part-time workers tend to get paid much less per hour (not just total), which intensifies the gap between full- and part-time workers. If "F" is a full-time worker who works 40 hours, and "P" is a part-time worker who works 20 hours, all other things being equal (if they work in the same field, same position, etc.), F probably makes more than twice as much as P.

(Here's a 2001 study that looks at some specific "problems" with part-time jobs. From the study's conclusion: "No matter how the differences are explained, most part-time workers take home disproportionately less pay, receive fewer benefits, and have fewer long-term career opportunities than their full-time counterparts with similar education, experience, and personal characteristics.")

Now, you know more than I do about empirical research methods, so please correct me if I'm wrong in any of this. I do think my post could use some editing/clarification based on your observation, but I still think the report is leaving something out.

John Althouse Cohen said...

(I've added an update in the middle of the post, with a link to these comments.)

LemmusLemmus said...

True, "hours worked per week" says nothing about who works part-time and who works full-time, but I wonder how interesting that divide is once you've controlled for hours worked. This raises the question how one would operationalize part- vs. full-time. I seem to remember once seeing a study in which simply everything under 25 hours was called "part-time" and everything else "full-time".

Having said that, it is of course possible that employees who work more hours per week also earn more per hour, controlling for other factors. As I didn't mention above (but can be seen from the same table on p. 42), the researchers controlled not only for hours worked per week, but, simultaneously, for hours worked per week squared. This functional form is perfect if (controlling for other variables) pay per hour rises linearly with hours worked. To the extent the actual data deviate from this ideal, the choice of an hours per week squared variable is less than perfect. For example, if there is a sudden "jump" as we move from 29 to 31 hours worked, the squared variable would not pick that up all that well. One could try other functional forms; whether this is desirable is a specialists' discussion.

Anyway, if you look at the right-hand side table of "figure 23" (actually a table) on p. 42, you'll see that the coefficient for the squared variable is significant, small and, surprisingly, negative. Which suggests that this kind of thing does not help to explain the pay gap between men and women; if anything, to the contrary (i.e., if you did not include this variable, the coefficient for gender would probably be a bit larger, not smaller).

More importantly, however, is that Sowell is right, of course. Just ascribing residual variance to your favourite factor, such as sexism, is the statistical equivalent of the God of the Gaps argument.