This tweet made me smile:
It was very funny, and anyone who has sat on the other side of a customer service call probably wishes for more “big bosses” to observe. But as a researcher I also found this to be a classic example of how you have to be very careful, when conducting market research, that you don’t influence behaviors just by being there. Clearly this boss still doesn’t really know what his staff is doing.
Of course researchers have all sorts of tricks for managing this dynamic, depending on the scenario. In my experience, the most important thing in preventing the impact of observation is to be aware that it happens. Simply asking, “What exists in this research environment that may influence behavior?” goes a long way.
I don’t know the background of @BAbackpacker’s situation, but if the “big boss” really wanted to understand the customer service dynamic, he could have recorded a wide range of phone conversations without letting the staffer know which specific ones would be reviewed or listened in, using a technology that was less obvious. Of course he could also hire a professional researcher who would be able to analyze the situation and create a more natural environment.