Hi John > On 6 Aug 2025, at 07:03, John C Klensin <john-ietf@xxxxxxx> wrote: > > Jay, > > Please take this as the constructive comment and suggestion it is > intended to be rather than as a complaint, Recommendations are always welcome! > but, for the future, > please think about saying something closer to "hear from people who > represent as broad a spectrum of IETF participants as possible" > rather than "...as many". I think you are conflating two related, but ultimately separate things. The first is the response set that we want and the second is what we say to get that response set. It is clear that within the IETF, as with many large organisations, an individual participant cannot accurately determine the spectrum of other participants, nor where they sit on that spectrum. That, after all, is why we run surveys. We could take an alternative approach of trying to identify a representative sample, but for surveys of this nature there are privacy concerns and it is a disproportionate amount of work. > If the survey gets answers from only a > limited and fairly homogeneous subset of the community (maybe even a > subset similar to those who actively participate in most or all > process-oriented WGs), it tells you what that group of people think > but, maybe, much less about the views of the overall community. And, > unless the size of that group is measured in the low thousands, it > doesn't tell you much about representativeness. I’m sure you know that the margin of error depends on the population and sample size. In general I aim for a margin of error of ~+/- 5% for these short surveys and ~+/- 2.5% for the annual survey. For a post-meeting survey we tend to get ~220 responses so a margin of error of ~+/- 6%. > That, in turn, > creates a challenge for which neither I nor the survey research > community have easy answers: how do you tell whether the survey > responses come from a representative sample of the community or just > a small, somewhat homogeneous, group and do so without questions that > would be intrusive and possibly problematic from a privacy standpoint > (the location, gender, and number of meetings attended questions that > are asked, help, but only a little bit. Most of our surveys include basic demographic questions and the answers to those can be used to compare response sets between surveys and with known facts. Previously (I can’t remember when) I had someone do a comparison of these demographics with our known meeting participant data and it was sufficiently representative at that time. thanks Jay > > thanks, > john > > > --On Tuesday, August 5, 2025 13:53 +1200 Jay Daley > <exec-director@xxxxxxxx> wrote: > >> This is a final reminder. >> >> Please take a few minutes to fill out our post-meeting survey for >> IETF 123 Madrid as this provides vital data to help us plan future >> meetings. We really need to hear from as many participants as >> possible: >> >> https://ietf.iad1.qualtrics.com/jfe/form/SV_bJZHjw5u4PnHA9g > > -- Jay Daley IETF Executive Director exec-director@xxxxxxxx
