Unless research groups remain in continuous contact, interaction and exchange communication with the rest of the stakeholders, their separation from the actual development and deployment process generates a dangerous discontinuity and an unnecessary gap between observation, design, deployment and redesign.

Artificial Biases

A diagnose, design or research group may not be representative of the public being served, introducing biases in the observations and conclusions. The bias can come in many different aspects, some often unnoticed:

• Social, cultural, ethnical, economical, professional, geographical, generational (age), sexual (gender), educational, political or even personal.
   
• A corporate person used to intranets, e-mail usage, regular meetings, top-down guidelines and strategies, periodic assessment and evaluation of individual achievements definitely has a different point of view, perception and paradigm than a prone to innovation action-focused social entrepreneur.
   
• A similar contrast is found between highly organized individuals with extensive academic formation and structured thinking and approaches and young students, as easily attracted to new ideas as distracted by anything else and with a facility to participate in unstructured, random and disorganized ways.

Limited Observation and Representation

A research group often deals with a limited sample of individuals in a controlled environment both of which are expected to represent the Universe to be served. Even when large scale and in-place observation is made, very seldom can the actual whole picture be documented. Several limitations are intrinsic to the intermediated observation approach:

Limited Representation

The sample of individuals may not always be representative of the whole. Just as the observation groups may introduce bias according to their own profile, even groups of individuals carefully selected often end up not being representative of the majority.

Think about it, look around your immediate environment and ask yourself: who is representative of others? Who is normal? Who, when observed and documented, would effectively represent your own preferences and behaviors?

Even when dealing with so called "personas" and made-up profiles, a true representation of users is not achieved or worst, unnatural combinations of aspects are done for the sake of inclusiveness. One quick validation technique can be to present these profiles to people and ask them if they identify themselves with any of these profiles and what differences they can point out from their own personal profile.

Partial Observation

An immense amount of issues that affect the users’ conditions is not registered when behavior patterns are isolated and observation is focused on users' responses to events and specific situations. Users’ decisions and preferences usually are part of a larger chain of events happening prior, during and after the focus of our observation and conditions that may change at random or cyclically.

Common examples are: work, business, study and family related cycles, having a bad day, the time of the month, the pressures of end of quarter financial reports, end-of-year efficiency evaluations, summer vacations, the stress of holidays, running out of funds as the date for the next paycheck gets closer, and of course unexpected things that happen everyday.

If you plan to serve just 1,000 people, you can count on at least one unexpected situation that affects the user’s behavior each day. That's 365,000 situations a year that the research or observation group will not document.

Peer Pressure and Social Trends

Users behave differently in an isolated environment or when facing situations individually. Isolated, partial observation fails to document the common peer pressure and social trends the community as whole deals with on a daily basis.

Altered Behavior and the Changing Hats Phenomenon

It is common knowledge that users also behave differently when they know they are being observed. But less well-known is the fact that when asked for opinion, comments or suggestions, people tend to put on a different hat and anoint themselves as consultant trying to provide elaborate or unnatural responses.

Much better results are achieved by focusing on the users' real actions and activities rather than on requesting focused descriptions or suggestions from users. So, instead of asking users how they would improve a product or process, more relevant responses can be generated by asking what did you do last week, yesterday or today, or how did you do similar things recently (not asking or revealing the actual subject we are addressing). This way the user can provide a depiction of an actual behavior and process rather than his/her personal dramatization of an ideal situation.

Focus on the Quality of the Observation and on their Documentation

It is also common for research groups to focus in the quality of the recommendations they make (for this is what they will be evaluated upon) and not in the end product or final results.