Measuring Impact … The Devil’s In The Details

Last week, some new numbers were announced regarding the impact that women-owned firms have on the U.S. economy. The Center for Women’s Business Research, with support from Walmart and the National Women’s Business Council, reported that women-owned businesses “contribute nearly $3 trillion to our national economy and create or maintain 23 million jobs.” While there is no disputing that looking solely at business revenues goes only part way toward measuring the full impact of a business in a community, exactly how to more fully account for the impact of women-owned businesses is a difficult question.
Bunny surprise
While the Center for Women’s Business Research has opened up an important discussion, they have not, in my view, added to the fabric of knowledge about women business owners and their enterprises in a wholly positive manner. The devil, as they say, is in the details.

Upon closer examination of the study, we observe that:

  • The estimates of economic impact are made using a purposive, non-random sample of only 417 businesses, conducted last year. A more complete set of usable data might have been better obtained from a larger, random-sample survey, or even from using Dun & Bradstreet data. One might also want to question the types of outputs computed in the model, and whether they are pre-recessionary or recessionary in their nature.
  • The model used was one developed to measure “the projection of secondary or multiplier effects from changes in direct sales in one industry within a study region. Multipliers can be estimated for a county, a state or for the entire nation (emphasis ours).” This appears to mean that a model developed for use in a single industry in a single region is being extrapolated to all industries over the entire country. This gives Womenable the heebie-jeebies.
  • While it is sexy to make comparisons between the revenues and expenditures of a company or group of companies with those of a nation, the inputs of the IMPLAN model used by the Center here and the components of Gross Domestic Product are likely very different. While the IMPLAN model is proprietary and not fully disclosed, it is most assuredly a case of “apples and oranges” to try to equate the impact of women-owned businesses with the GDP of nations.

In the spirit of full disclosure, I’m speaking here not only as the President and CEO of Womenable and a member of the women’s business community, but as the former Managing Director and Director of Research of the Center for Women’s Business Research and former Executive Director of the National Women’s Business Council. So, while it pains me to call out my colleagues, I do so in the spirit of making sure that the users of this new information are more fully aware of from whence it came, and with the hope that future efforts along these lines are more methodologically rigorous.

Some may feel that imperfect information is better than none at all, but Womenable believes that unassailable information – provided with more apt comparisons and fuller disclosure – does the women’s business community a much better service than does information that overreaches; that goes too far out on a limb without the underpinning of a more solid methodological framework.

As we all know, once information is out there it can take on a life of its own. While Womenable applauds this study’s goals and supports the overall message that accounting for the impact of women-owned firms should go beyond counting firm revenues (and, indeed, beyond counting currency spent by firms and their employees), we do not back this study’s execution nor its numerical conclusions.

We welcome a dialogue on this issue, and are willing to be proven too cautious. What say you all?

2 thoughts on “Measuring Impact … The Devil’s In The Details

  1. Julie,

    I really appreciate you blogging about this… I would not have ‘looked under the covers’ on this one, I would have taken it for granted and trusted the design and ran with the findings. Your point(s) are well taken. You are a valuable asset to us! I am curious, based on what you see, what would you like to see next?

    Lorin

  2. Lorin, thanks for the comment. I’ve raised the issue not only to wave a ‘caveat emptor’ flag – to alert folks to make sure they understand the basis upon which estimates such as these are made – but to help start a wider discussion about how we in the women’s business community can better highlight the contributions that women-owned firms are making. I think they go well beyond the economic. In this day and age, as people have grown less trusting of the financial ‘fat cats,’ we have an opportunity to show that thinking more broadly beyond the bottom line is not only profitable but better for society as a whole.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s