What we really think
In pursuit of its master plan for the Vineyard, the Martha's Vineyard Commission members and their planning committees have used a variety of surveys to gather the views of Islanders, their seasonal neighbors, and visitors about what they'd like the Vineyard to become. There are more surveys ahead. In making such surveys, distributing questionnaires or making them widely available is a common enough method, and it arises from a commendable determination to take the planning lead from the people who will inherit the blueprint. But, this survey approach to gathering the views of one's subjects toward the work one is doing on their behalf is flawed, and ultimately fruitless.
Naturally, everyone has opinions. Some offer their views free for the asking. Some need no invitation at all. Most people never get asked and never volunteer. Some opinions are worth having, and others are worthless. The same is true of surveys and polls to which many of us respond. Of course the planners want to know what we think about their work and our future, but because they intend to use our responses to their questionnaires to make public policy decisions, we need to consider the survey results with a sharply critical eye. Deciding when a survey is authoritative, or when it is interesting but inconclusive, or even meaningless, is extremely important.
Recently, in an Essay that appeared on The Times OpEd Page, Jim Athearn, chairman of the Island plan steering committee, invited Islanders, and anyone else interested, to participate.
"Public involvement," Mr. Athearn wrote, "will be sought in a wide variety of ways, including public forums, workshops, and surveys, as well as creation of work groups on specific topics and a large network of citizen advisors.
"We are starting out by asking some fundamental questions about how Vineyarders see the future of the Island, such as the following (expressed somewhat provocatively as a choice between two extremes to spark discussion)."
The business of measuring the views of any group toward anything is complicated. The results may be depended upon only if the inquiry is made in a statistically defensible way. The preparation of the instrument that will be used to gather opinions is critical, of course. The questions - how they are cast - is fundamentally important. For instance, Mr. Athearn, admitting the provocative flavor of the question, proposes, "Should we allow anyone who wants to live on the Island to move here, or should we strictly limit population growth and development?"
Does this mean "anyone" in the sense of "how many", or in the sense of "what sorts"? Does "limit population" have to do with numbers or types? Is either governable by local rules, or are there constitutional freedoms implicated here? Or, are questions such as these to be construed by the self-selected respondents any way they like, and in as many different ways as there may be responses?
Just as important, who are the folks whose views will be gathered by such a survey instrument? After all, "To fill out the survey," Mr. Athearn wrote, "go to the web site www.islandplan.org or any Island library." But, who will the respondents be? To what degree will they represent the rest of us in the Islander/seasonal Islander/Island visitor demographic? What reason will there be to rely on the responses at all?
This page has often urged public organizations, with a desire or a need to learn what people think, to employ paid, professional, disinterested polling organizations to help develop polling instruments and then to make a scientifically sound inquiry. Scientifically valid surveys require representative samples, whose participants are carefully identified and approached for their responses.
According to the National Council on Public Polls, "The results of the well conducted scientific poll provide a reliable guide to the opinions of many people in addition to those interviewed - even the opinions of all Americans. The results of an unscientific poll tell you nothing beyond simply what those respondents say."
And, they may tell you only what you wanted to hear.
The MVC's Island planning effort ought to be based on reliable data of every sort, including authentic polling data, scientifically developed. Otherwise, survey results may be regarded as interesting, perhaps, but meaningless.