Measurement construct validity is especially vulnerable. By definition, participant observation does not use standardized tests. As a result, observations have a special proneness to random measurement error and, thus, unreliability.
Measurement validity cannot exceed measurement reliability. By their non-quantitative nature such observations do not lend themselves to reliability estimates. Because the participant observer often works in varying circumstances and with varying categories, we cannot assume she or he has good consistency over occasions.