Graham Lamburn, Technical Manager at Powerwatch UK, provided an interesting comment on the usefulness and validity of data obtained in psychology provocation studies.
***********************************
Graham Lamburn on psychology provocation studies
This has always been one of the trickiest aspects to even supposedly double blinded EHS experiments.
Firstly, there is no guarantee that those in the “sensitive” group are indeed sensitive, and no guarantee that the control group are not sensitive. I’m only aware of one of the provocation studies that addressed this issue, and it didn’t make any adjustments due to their findings on the basis that they wouldn’t have had enough people left for significance. it takes only a very small proportional percentage of people assigned into the incorrect groups to make statistical significance very hard to achieve even under the most dramatic experimental outcomes.
Secondly, people claim to exhibit sensitivity to very different electromagnetic stimuli. It is not easy to assess whether this is plausible when we don’t understand any mechanisms by which their symptoms are supposed to be triggered, but it does add yet another confounder when “sensitive” individuals are being exposed to signals that they have not necessarily claimed to have a sensitivity to.
Thirdly, there’s the issue of how long after exposure the sensitivity is supposed to have taken affect, which also seems to be anecdotally very variable.
Fourthly, as you mentioned above, collecting all the data as entirely subjective data makes it unreliable at best and close to useless at worst.
It is a shame that so many of the provocation studies are seen as some gold standard just because they’re double blinded in a standard case-control exposure design, when they are nothing of the sort. I have not yet seen a single EHS provocation study that I would have confidence in producing a result that really moves the subject forward – either not having the resolving power, insufficient effort to control for the various possible confounding factors, or just simply inadequately collected data to allow for robust analyses and conclusions to be drawn.
Having said all that, designing a truly excellent provocation study is an exceptionally difficult task. It may not be possible to do a well designed and conclusive case control study. I still fail to see why it wouldn’t be a “better” place to start to have a single self-diagnosed sensitive individual and a single matched control, tested in the exact situation to which the sensitive participant claims to react, with a “true” sham setup, over hundreds of live and sham tests to gain some level of statistical resolving power. Biological markers and immune system response (C-reactive protein maybe?) would be an even more convincing step as they produce objective data.
As Sir William Stewart said many years ago, the first stage is to produce compelling evidence in a single individual that the sensitivity exists. Working out how many people might be affected, what range of different exposures people can be sensitive to, what symptoms can be expected for those who are sensitive, etc etc, are all “next steps” in the science in my opinion.
BRHP - Between a Rock and a Hard Place 
“I still fail to see why it wouldn’t be a “better” place to start to have a single self-diagnosed sensitive individual and a single matched control” … “over hundreds of live and sham tests to gain some level of statistical resolving power. ”
My answer would be: because that could make sensitive people so extremely sick that it is unsure if they ever recover at all from the strain. A number of sensitive persons report that symptoms are only reposable until a certain point of exposure. It would be highly unethical to go over that tipping point during an experiment.
Indeed, this is the statement I wished to see for a long time, only better stated than I had thought it out myself:
“I still fail to see why it wouldn’t be a “better” place to start to have a single self-diagnosed sensitive individual and a single matched control, tested in the exact situation to which the sensitive participant claims to react, with a “true” sham setup, over hundreds of live and sham tests to gain some level of statistical resolving power.”
As a student of social anthropology, as well as somewhat more “exact” sciences, the situation as to research on EHS gives me flash-backs to the methodological challenges in doing a social anthropological fieldwork in a foreign tribe: There is a high, but unknown number of relevant parameters, and no way of sorting them out or testing their relevance ahead of the study. Hence, the student relies on observations of individual cases and a stepwise developed ability to observe and describe a pattern, often still without understanding the mechanisms creating it, nor with any way to confirm the generality of it through precise statistics.
Not understanding the mechanisms, nor the incidence, is neither in social anthropology nor in medicine a valid reason for claiming that the pattern does not exist at the individual level or in parts of the society in question.