Statisticians/pollsters -- correct design of a poll of N respondents

Statisticians/pollsters -- correct design of a poll of N respondents

This is a discussion on Statisticians/pollsters -- correct design of a poll of N respondents within the The Second Amendment & Gun Legislation Discussion forums, part of the Related Topics category; For you statisticians and pollsters out there, those who specifically know how to effectively design a poll to get a truthful and correct representation of ...

Results 1 to 10 of 10
Like Tree13Likes
  • 3 Post By ccw9mm
  • 2 Post By buckeye .45
  • 1 Post By high pockets
  • 2 Post By Longstreet
  • 1 Post By NONAME762
  • 1 Post By ccw9mm
  • 2 Post By Goat_Guy
  • 1 Post By BenGoodLuck

Thread: Statisticians/pollsters -- correct design of a poll of N respondents

  1. #1
    VIP Member Array ccw9mm's Avatar
    Join Date
    Jul 2006
    Location
    USA
    Posts
    27,003

    Question Statisticians/pollsters -- correct design of a poll of N respondents

    For you statisticians and pollsters out there, those who specifically know how to effectively design a poll to get a truthful and correct representation of results ...

    Q: How would you go about designing a poll of citizens in three large states in the USA, if you needed to determine the actual, true support for a given concept? How would you effectively guard against bias of all sorts, and all the other problems a poll can have if poorly designed? How would you craft the questions, to get at the true beliefs instead of how you imagine they'd belief if you could twist the questioning? How would you ensure the polling sample was sufficient to effectively and honorably extrapolate the results to imply the larger population did indeed feel that way? etc

    For those who aren't certain, or who don't work in such a field where designing quality polls is what you do, move along.


    Notes:

    Inspired by the discussion going on elsewhere in the 2A forum, on recent polling in NY, NJ, VA that proclaimed a majority back the idea of national gun registration (click).

    Am not looking for an intro course in statistics and poll design. Am simply thinking a high-level, concise summary of the key design points might go a long way to help folks understand why a well-crafted, honorable poll design can indeed give a clue as to a larger population's opinions.

    Please avoid the 'bad/twisted polling techniques' harangues, in this discussion. Let's keep the eye on the ball, focusing strictly on the question of effective, statistically-justifiable and -relevant polling to get the task done. The task being a truthful representation, not whatever ulterior motives might or might not exist in those commissioning or executing the poll.

    .
    Your best weapon is your brain. Don't leave home without it.
    Thoughts: Justifiable self defense (A.O.J.).
    Explain: How does disarming victims reduce the number of victims?
    Reason over Force: The Gun is Civilization (Marko Kloos).
    NRA, SAF, GOA, OFF, ACLDN.


  2. #2
    Moderator
    Array buckeye .45's Avatar
    Join Date
    Nov 2006
    Location
    Ohio
    Posts
    7,682
    You have to make sure that the polling population is actually representative of the actual population.

    The simplest way to do that is to poll the entire population (although not a resource effective method).

    But, without a representative population, any results you get are going to be skewed.

    Then you can get into avoiding introducing biases through how the questions are asked, or through situational means.
    NONAME762 and gatorbait51 like this.
    Fortes Fortuna Juvat

    Former, USMC 0311, OIF/OEF vet
    NRA Pistol/Rifle/Shotgun/Reloading Instructor, RSO, Ohio CHL Instructor

  3. #3
    VIP Member Array high pockets's Avatar
    Join Date
    Jun 2011
    Location
    Georgia for now
    Posts
    4,646
    Your parameters preclude an answer. I have never seen, or used, a poll that was not slanted, or designed, to target a specific group.
    gatorbait51 likes this.
    "If you make something idiot proof, someone will make a better idiot."

    - Anon

  4. #4
    Ex Member Array Longstreet's Avatar
    Join Date
    Jan 2014
    Location
    Indianapolis
    Posts
    741
    My answer would not resonate with anyone other than those who are highly educated in such things (it's far more complicated than one could explain in an Internet forum.) Generically, I can tell you that such "polls" are almost always nothing more than gimmicks by a special interest group with an agenda. Even the supposed professoinal pollsters, Zogby, Rasmussen, etc., have agenda-driven leanings that render their "poll" results biased and, in many case, statistically irrelevant.
    Last edited by Longstreet; March 5th, 2014 at 02:41 PM.
    NONAME762 and gatorbait51 like this.

  5. #5
    VIP Member Array NONAME762's Avatar
    Join Date
    Feb 2013
    Location
    About 235M out of The Palouse WA
    Posts
    7,748
    Just one quick thing for the OP here.

    It is not often I will answer a poll question even here in this forum. My reason is I prefer to think long and hard before I answer. Many poll questions are written in such a way that a 'yes' answer can be interpreted as a 'no' answer.
    gatorbait51 likes this.
    Firing a suppressed is on my Bucket List.

    I'm just a spoke in the wheel but not a big deal.

    America...a Constitutional Republic. NOT a democracy as the liberals would have us believe.

  6. #6
    VIP Member Array ccw9mm's Avatar
    Join Date
    Jul 2006
    Location
    USA
    Posts
    27,003
    Quote Originally Posted by high pockets View Post
    Your parameters preclude an answer. I have never seen, or used, a poll that was not slanted, or designed, to target a specific group.
    Polls by definition aren't the whole population, so in effect the mere fact we're speaking of polls precludes a perfect poll or answer on how to create one. Agreed.

    What I'm getting at is simply what the key poll design techniques would have to be to come as close as any poll could come (realistically) to honorably extrapolating to the larger population. Figured there are a few "data heads" and statisticians on the forum who'd care to present a handful of key thoughts on the design aspects.
    gatorbait51 likes this.
    Your best weapon is your brain. Don't leave home without it.
    Thoughts: Justifiable self defense (A.O.J.).
    Explain: How does disarming victims reduce the number of victims?
    Reason over Force: The Gun is Civilization (Marko Kloos).
    NRA, SAF, GOA, OFF, ACLDN.

  7. #7
    Member Array Goat_Guy's Avatar
    Join Date
    Dec 2012
    Location
    Iowa
    Posts
    125
    You are referring to obtaining an unbiased, random sample. There are many methods available but whichever is used, the idea is that every element (member) of the population has an equal chance of selection - easier said than done. A telephone survey might employ a computer which randomly dials numbers for example. But there is more to consider: what time of day or when in the week do you call? Also, how a question is phrased can bias the results, especially if there are multiple questions. Should the question be open or closed response? Issues like these need to be dealt with in a good survey.

    Legitimate polling companies - Gallup for example - go to considerable effort to obtain an unbiased survey. It can be done, but it isn't cheap - which is why newspaper, television, and internet web site polls are generally useless. They don't have the know-how or money to do a good job.

    Incidentally, a fairly small sample is often all that is needed if the survey is done right. Only a few hundred responses can adequately represent a population of millions. This seems unintuitive, but it is true. A larger sample will reduce the margin of error but will not help if the results are biased.
    gatorbait51 and maxwell97 like this.
    Never get out of the boat....

  8. #8
    VIP Member
    Array BenGoodLuck's Avatar
    Join Date
    Aug 2005
    Location
    Texas
    Posts
    3,147
    I found the information here to be helpful: Analyzing polls-Interpretive analysis-Questionnaire design

    Btw, you didn't provide any answer choices to your question!
    gatorbait51 likes this.
    Ben

    Cogito, ergo armatum sum. I think, therefore I am armed. (Don Mann, The Modern Day Gunslinger; the ultimate handgun training manual)


  9. #9
    Senior Member Array bklynboy's Avatar
    Join Date
    Jun 2006
    Location
    VA
    Posts
    580
    Assuming the questions have been posed in an unbiased fashion, many other factors can affect the randomness of the sample. The poster above mentioned the time of day that the call was placed. There are other factors as well. Modern pollsters have found that due to the fact that landline owners tend to skew older than mobile owners, that may skew results as well. Different social media can also skew the results. The bottom line is that in order to assure that your results are random, you need to collect those results in multiple runs and in different ways and then run a test to test for randomness. The most commonly used test for randomness is Chi Square test or a Pearson test. If the sample was truly random, then the Chi Square test will confirm it

  10. #10
    Distinguished Member
    Array oldskeetshooter's Avatar
    Join Date
    Nov 2012
    Location
    Georgia..South of the mountains, north of the sand flies.
    Posts
    1,537
    Being in the Automobile business for almost 40 years, I have seen many companies conduct polls. Most call them "Surveys". Yes, they are written by different companies for different auto manufacturers. Sometimes, the most open questions, with the greatest amount of variables are the ones that the manufacturers base their analysis on. The question "Would you recommend this car to someone else?" is more important, to the manufacturer, than any other. A "No" answer will generally get the customer another letter sent to them. "Please, explain why you would not recommend this car to someone else". This second letter really let's the manufacturer know how the customer feels and what needs to be done to improve the product.
    A wise man once said: "Bugout bag?..What's that? Is that all the junk you sidewalk commandos plan on humping when the SHTF...I'll grab a Nylon 66, a box of 22s and a poncho liner and in less than a week I will have all of your stuff and everything else that I need for the duration."

Links

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  

» Log in

User Name:

Password:

Not a member yet?
Register Now!

» DefensiveCarry Sponsors