Detection of "curb-stoning" in surveys
Overview/Narrative
When surveys are taken, the situation can arise where one or more of the persons taking the survey do not actually interview a person, but instead, make up the data. There are a number of reasons why this may happen - e.g. time pressure to complete the results, inconvenient to visit, too far to drive, too late in the day, etc. The industry term for this is "curb stoning", based upon the surveyor completing the survey while sitting on a curb stone.
It is essential to detect when instances of this may be happening, as the data is made up, it can invalidate the survey study results. One means of detecting potential instances of curb stoning is through the application of Benford's Law, which provides an expected distribution for the frequency of first digits. This law has been found to apply to many numerical results, such as lengths of rivers, number of people residing in a household, etc.
Audit Objective
The objective is to identify potential instances of curb stoning based upon a comparison of the survey data completed compared with that which would be expected, if the data distribution conformed with Benford's Law. Note that some elements of a survey will NOT comply with Benford's law where they are of a predetermined range. However, in this case, all the data elements should conform - i.e. Street address number, number of people in the household, and the distance to the nearest church.
Audit Procedures / Audit Program
1. Obtain survey data in electronic format, note data elements and state whether they would be expected to conform with Benford's law.
2. Sort the data by surveyor.
3. For each surveyor and each data element expected to conform with Benford's law, prepare a population distribution and a chart, using the first digit.
4. For each surveyor and each applicable survey element, compute the D-Statistic using the Kolmogorov-Smirnov metric.
5. Review each chart and data table prepared to determine if the data appears to conform with Benford's Law.
6. For data which does not appear to conform, how does the D-Statistic compare with the others?
7. Write a report explaining your observations, and provide conclusions and recommendations.
Further details on detection of curb-stoning.
Comments welcomed in Audit blog
|
Sponsors
HOT STORIES
