It should be no surprise to anyone who has done their homework that the United States incarcerates the largest number of individuals per capita.1 It is also not a surprise that black individuals make up the largest single demographic percentage of these individuals significantly outpacing their per capita population relative to other race and ethnicities.1 Individuals when discussing the nature of the criminal justice system frequently cite statistics to validate this racial/ethnic disparity. Typically there are two types of responses by most individuals when exposed to these statistics depending on personal perspective: 1) Currently the criminal justice system is unfair to black individuals; 2) black people commit a disproportionate amount of the prosecuted crime. Interestingly enough most people seem to think that these two rationalities are mutually exclusive because rarely does anyone cite both when discussing how blacks and the criminal justice system interact. The question is which of these two rationalities is the chief governing factor behind the incarceration rate for blacks in the United States?
It would not be surprising if at this moment a number of the individuals who prescribe to the first school of thought taking offense to the very possibility of legitimacy for the second rationality, which goes to show the emotional reality of this issue. The chief problem with individuals who lament the number of blacks in prison is that they avoid asking whether or not those individuals actually broke the law and are in jail for legitimate reasons. While there certainly are individuals who have been denied justice and are incarcerated on fraudulent grounds for crimes they did not commit, the simple fact is that a vast majority of individuals, regardless of race or ethnicity, are in jail because they were appropriately convicted a crime.
Addressing the last sentence, realistically there are five explanations for the disparity between incarceration rates of blacks and those of other races/ethnicities:
1 - These individuals are actually committing crimes and are legitimately getting caught supporting the above contention that blacks commit a disproportionate amount of the criminal activity in the United States.
2 - Blacks only commit a small amount of the total crime in the United States, but are less able to conceal their criminal activity, thus their demographic is disproportionally represented in the incarcerated population versus the total number of crimes that are actually committed; this rationality supports neither of the above initial viewpoints.
3 - Bias actively leads the criminal justice system to pursue charges against crime committing black individuals versus crime committing individuals of other races and ethnicities when available evidence is significant in all scenarios supporting the position that the criminal justice system is currently unfair to blacks.
4 – Blacks receive unjustified jail sentences that exceed sentencing guidelines set forth for the associated committed crime supporting the position that the criminal justice system is currently unfair to blacks.
5 - A disproportionate percentage of jailed blacks are innocent of the convicted crime; whether racism played a role in that fraudulent conviction is unclear, but probable for a number of them supporting the position that the criminal justice system is currently unfair to blacks.
The third reason differs from the second reason because of the actions of the individual committing the crime relative to the actions of law enforcement agencies. For example the second reason could be invoked in a situation where a black individual shoots someone in the middle of a neighborhood with numerous witnesses available to testify where a non-black individual shoots someone in a private residence when there are no witnesses, thus there is significantly less evidence to promote an arrest or a conviction. The third reason could be invoked in a situation where the circumstances and scenario of the criminal behavior are similar, but law enforcement agents pursue charges against the black individual instead of the non-black individual. Of course a final point must be made in that for all reasons other than the last one the black individual did actually commit a crime, thus one should not argue that this individual is inappropriately incarcerated.
It is important to consider for the statistics that are frequently cited that suggest racism in the criminal justice system the lopsided nature of non-violent drug offenses. Individuals who use and/or sell illegal drugs make up the largest number of incarcerated individuals (for a specific crime) and it is this crime that produces the most significant portion of the disparity between incarcerated blacks and those of other races/ethnicities. Based on this disparity numerous individuals/groups have claimed that non-violent drug offenses are evidence of racism in the criminal justice system. Unfortunately for a vast majority of these individuals blindly citing the statistics is as far as they go in their analysis. Recall what Mark Twain once said, “There are three kinds of lies: lies, damned lies and statistics.” Without understanding the origins and the “why” behind the raw data that create the statistics, using statistics to argue for a certain perspective is inappropriate and foolish.
With regards to the issue of black incarceration rates a chief point is whether or not drug related crimes are bias against blacks (or to a larger extent minorities in general). However, it is up to those who believe this characterization to prove it; i.e. the burden of proof is on those individuals to demonstrate that drug laws are bias against minorities. There are certain issues that must be addressed by these proponents outside of simply citing statistics.
First, one must analyze whether or not minority users are being sent to jail due to a higher wrongful conviction rates than white users not just arrested at a higher rate despite the arrests being appropriate. To justify this conclusion one would have to conduct an analysis that demonstrated more aggressive incorrect convictions for minorities. For example in county A consider that there are 100 white and 100 black people, 80 black people are accused of violating drug laws with 75 being rightfully convicted and 5 being rightfully acquitted versus 40 white people being accused of violating drug laws with 37 being rightfully convicted and 3 being rightfully acquitted. In this scenario there is no racism as the conviction rates are similar, black drug use is simply higher than white drug use. In a county B consider that there are 100 white and 100 black people, 50 black people are accused of violating drug laws with 45 being rightfully convicted and 5 being rightfully acquitted versus 50 white people accused of violating drug laws with 5 being rightfully convicted and 5 being rightfully acquitted and 40 being wrongfully acquitted.
In the second scenario one would argue racism because the justifiable conviction rate is skewed so much in favor of blacks and typically whether or not an individual is guilty of a drug offense is rather simplistic (i.e. there is little room for subjective rationality or interpretation). Unfortunately those arguing racism must address the issue of unequal justice between economic classes. Despite the contrasting ideological belief in the judicial system, it is widely understood that empirically the poor receive less equitable treatment in the legal system than the rich and a larger percentage of minorities are poor. Therefore, to prove racism in the execution of drug-based court convictions one has to identify a wrongful conviction pattern and then untangle the web of bias between race/ethnicity and economic standing, a difficult task.
A second issue that must be addressed is analyzing the second and third points above by looking at how different races violate drug laws. For example initially when looking at the available information for marijuana arrest rates one could argue in favor of racism in that minorities are arrested at a disproportional rate than whites for drug possession despite similar usage rates, or even higher usage rates by whites (depending on what type of polling information is used). However, this accretion of racism hits a snag when considering how the crime is committed. Middle class and rich individuals, more often white, have resources available to them to make their illicit drug use more evasive than less wealthy individuals. It is inappropriate to suggest that a law is racist if one group has less ability to evade it than another group when there is no selective enforcement intent. Committing a crime in a public area and then being arrested and convicted for it cannot be viewed as selective targeting in any reasonable way.
A third issue that is imperative to making a claim of bias in the enforcement of drug laws is whether or not the law itself is bad. Unfortunately an argument that drugs laws are bad cannot be made as an element of necessity. Individuals that are convicted of various drug crimes are not akin to Jean Valjean stealing bread for his sister’s starving child. One does not need to consume various illicit drugs to survive nor does the consumption of these types of drugs produce unique positive effects that cannot be otherwise derived through legal means. It is also difficult to argue this point rationally on the basis of race with respect to stating that just because one group of individuals are convicted of a given crime that the crime is racist. If this logic were sound then one could argue that if a majority of individuals convicted of embezzlement were Jewish then embezzlement is a bias law.
Based on these three elements of that have yet to be proven one cannot accurately argue that drug laws are racist simply because a lot of black individuals are convicted. In reality a vast majority of black individuals commit a criminal offense involving drugs and are appropriately convicted for that violation. Perhaps one can attempt to rationally argue that certain drugs laws involving simple possession have too strict a penalty from a relative standpoint of their negative influence on society, but as it stands one cannot make that argument on grounds of simple racism or other bias.
That said it would be understandable to move from the issue of crippling bias in their execution, there is the question of whether or not drug laws carry the appropriate punishment. Setting aside mandatory minimums because most people misrepresent their application due to confusion between associated violence and quantity of drugs possessed, some argue that bias exists in habitual offender laws that mandate harsher sentences for repeat offenders. The problem with making this argument is that repeat offenders are not deterred from their criminal behavior by the same level of penalty or certainty of punishment previously accepted hence why they committed the crime again. Individuals commit crimes in order to produce some form of advantage in life. Most individuals either out of concern for the associated punishment or through general positive morality do not commit crimes. However, obviously some individuals are not concerned about the base severity of the punishment or its certainty because they actually engage in criminal behavior. Therefore, what should be the response if an individual continues to violate the law?
It is difficult to argue for the decriminalization or penalty reduction for certain laws simply because one demographic is unable to conceal their violation of those laws. However, some people seem to argue exactly that, but would that strategy actually solve the problem? While a number of minorities, including blacks, are incarcerated for drug crimes one particular demographic of blacks are missing from jail cells, well-off or rich blacks. Rarely does an upper-middle class or rich black person go to jail for simple drug possession, thus most of the blacks in jail for drug possess are low income. What happens to these individuals in a world where drug use is legalized? A number of addicts are unable to identify that they have a problem with drug use, therefore, if the law is unable to “reach” these individuals what will ever stop them from abusing drugs?
While it can be argued that certain laws, most notably some drug possession laws, could be better addressed by court ordered drug rehabilitation versus incarceration, individuals who reference the criminal justice system as racist tend not to make this suggestion. As mentioned above these individuals are so distracted by the number of black individuals in jail that they forget that a vast majority of them actually did break the law they are in jail for. A better strategy would be to decriminalize minor drug possession from any felony to misdemeanors forcing repeat violators to seek treatment or accept incarceration. Some argue for the exact system utilized by Portugal, but those individuals must understand the difficulty of this idea by appreciating the logistics difference between enforcement in the U.S., a country with over 300 million individuals, and enforcement in Portugal, a country with around 10 million individuals.
The best thing individuals can do to help drug users appears to have two prongs: 1) ensure the proper measures are available to identify improper drug use and assign these individuals to appropriate treatment arenas; 2) petition for the passage of a guaranteed basic income (GBI) to ensure that low income individuals have the resources to effectively recover and stay recovered from any drug addiction.
Overall drug law enforcement is not racist and because most of the prison demographic disparity occurs through drug laws, the disparity itself is not racist. If one wants to argue for a different way to respond to those who violate certain laws over simply throwing the individual in jail that argument needs to be done logically not through inaccurate over-emotional race baiting because while on a whole the criminal justice system is not perfect, blindly proclaiming it racist is foolish.
Citations –
1. Carson, A, and Golinelli, D. “Prisoners in 2012 – Advance Counts.” Department of Justice. July 2013. http://www.bjs.gov/index.cfm?ty=pbdetail&iid=4737
Showing posts with label Statistics. Show all posts
Showing posts with label Statistics. Show all posts
Friday, June 27, 2014
Monday, June 21, 2010
Brief Discussion of Precision Statistics
This blog has previously discussed the importance of using statistical analysis when making decisions and analyzing information. However, one point that was not made was that both the information used in the analysis and the methodology of the analysis need to focus on generating meaningful conclusions. Without meaningful conclusions the analysis itself is rather worthless and may even lead people to misunderstand the power of statistics.
For example a very simple example of the real descriptive power of statistics can be taken from analysis of possession percentage from soccer (football). Ball possession is often considered one of the more important statistics because it typically describes which team is controlling the flow of the game. However, the description of control is extremely broad. If possession was divided between the offensive and defensive half then statistical analysis is significantly more powerful in generating an understanding of the general behavior of the game. If the new possession statistic demonstrates a large amount of total possession, but most of it in the defensive half then without watching any tape one can reasonably anticipate that such a team uses a long-ball based offense and pushes extra bodies back on defense. Interestingly soccer has already demonstrated the power of deeper statistical analysis where stats are taken of not only of shots on goal, but also how many of those shots are on target illustrating the overall effectiveness of the shots attempted.
One may argue that using such a simplistic example does a disservice to the importance and power of precision statistics, but most people inherently shy away from statistics and would probably not appreciate and/or understand more eloquent and complex examples. Therefore, when dealing with statistical neophytes it is important to introduce precision statistics through a medium that these individuals will care about, thus motivating an attempt to understand driven to better their own knowledge as a means to better enjoy the medium. Basically killing one bird with two stones. Overall not only is it important to include statistics in any deterministic analysis, but the representation of those statistics needs to be appropriate both in accuracy and depth so they tell the important parts of the story.
For example a very simple example of the real descriptive power of statistics can be taken from analysis of possession percentage from soccer (football). Ball possession is often considered one of the more important statistics because it typically describes which team is controlling the flow of the game. However, the description of control is extremely broad. If possession was divided between the offensive and defensive half then statistical analysis is significantly more powerful in generating an understanding of the general behavior of the game. If the new possession statistic demonstrates a large amount of total possession, but most of it in the defensive half then without watching any tape one can reasonably anticipate that such a team uses a long-ball based offense and pushes extra bodies back on defense. Interestingly soccer has already demonstrated the power of deeper statistical analysis where stats are taken of not only of shots on goal, but also how many of those shots are on target illustrating the overall effectiveness of the shots attempted.
One may argue that using such a simplistic example does a disservice to the importance and power of precision statistics, but most people inherently shy away from statistics and would probably not appreciate and/or understand more eloquent and complex examples. Therefore, when dealing with statistical neophytes it is important to introduce precision statistics through a medium that these individuals will care about, thus motivating an attempt to understand driven to better their own knowledge as a means to better enjoy the medium. Basically killing one bird with two stones. Overall not only is it important to include statistics in any deterministic analysis, but the representation of those statistics needs to be appropriate both in accuracy and depth so they tell the important parts of the story.
Friday, January 29, 2010
Reevaluating Quarterback Rating as a Performance Tool
In football the quarterback rating statistic has always been a quirky point of emphasis. It is one of the chief indicators that pundits use to evaluate the performance of a quarterback in a given situation be it when blitzed, in the 2 minute drill, on the first drive of the game, etc. However, despite all this attention paid to the quarterback rating, it can be argued that the interpretation of the statistic itself is in error. Most view the meaning behind the quarterback rating as ‘the efficiency of a quarterback’. Although useful, a better statistic would be to evaluate the influence of the quarterback in relation to that efficiency instead of focusing on efficiency alone. Quarterback rating should judge the prolific nature of the quarterback if it is to be an effective quantitative tool in quarterback performance measure.
The formula used to compute quarterback rating (shown below) was developed by Pro Football Hall of Fame executive Don Smith in 1971.
The equation breaks down into 4 separate components: first, completion percentage where 50% was used as the average benchmark. That is an average quarterback performance involved completing 50%. From that base point poor and high quality performance points were established at 30% and 70% respectively. The second part involves yards per attempt with an average performance being 7, poor being 4 and high quality being 11. The third and fourth parts focus on touchdown passes per attempt ratio with an average performance being 5% and interception per attempt ratio with an average performance being 5.5%. Overall an average performance netted 1 point whereas poor and high quality performances netted 0 and 2 points respectively. Finally the 100 divided by 6 element was based on an average performance netting a 66.7% out of 100% grade scale. Note that 2.375 is the highest total allowable for any of the components.
The problem with the above methodology as it relates to what should be the goal of the quarterback rating is that the efficiency measure is phantom exponentially extended. Basically the methodology projects a performance ad infinitum based on the current statistics. It is due to this inherent application why Quarterback A can complete 8 of out 10 passes for 168 yards with 2 touchdowns and 0 interceptions (statistically perfect rating 158.3) and have a better rating than Quarterback B who completes 32 out of 41 passes for 410 yards and 4 touchdowns and 1 interception (130.7). Quarterback A had a more efficient performance than Quarterback B, but which quarterback was more instrumental in the offense of their particular team? Clearly Quarterback B, but the quarterback rating does not accurately reflect that reality.
Therefore, if quarterback ratings are going to continue to be used as quantitative measurement tools in quarterback evaluation, a cap needs to be assigned to curtail the inherent exponential proficiency estimation. The best means to determine influence would be relate the cap back to yardage. The following criterion or something similar would be suitable:
Yards = Maximum Quarterback Rating Possible
0-199 = 99.9
200-249 = 119.9
250-299 = 139.9
300+ = 158.3
With the application of these caps the overall formula for calculating quarterback rating would not change, but if a quarterback failed to throw for more than 249 yards it would not matter if the formula calculated a rating of 146.7 because officially the rating would be reduced to 119.9. Moving quarterback rating beyond simple efficiency and adding game influence increases its statistical power and the ability to accurately differentiate between high quality and quarterbacks that are only asked to do so much to aid their offense, which is supposed to the real point behind quarterback rating in the first place.
The formula used to compute quarterback rating (shown below) was developed by Pro Football Hall of Fame executive Don Smith in 1971.
The equation breaks down into 4 separate components: first, completion percentage where 50% was used as the average benchmark. That is an average quarterback performance involved completing 50%. From that base point poor and high quality performance points were established at 30% and 70% respectively. The second part involves yards per attempt with an average performance being 7, poor being 4 and high quality being 11. The third and fourth parts focus on touchdown passes per attempt ratio with an average performance being 5% and interception per attempt ratio with an average performance being 5.5%. Overall an average performance netted 1 point whereas poor and high quality performances netted 0 and 2 points respectively. Finally the 100 divided by 6 element was based on an average performance netting a 66.7% out of 100% grade scale. Note that 2.375 is the highest total allowable for any of the components.
The problem with the above methodology as it relates to what should be the goal of the quarterback rating is that the efficiency measure is phantom exponentially extended. Basically the methodology projects a performance ad infinitum based on the current statistics. It is due to this inherent application why Quarterback A can complete 8 of out 10 passes for 168 yards with 2 touchdowns and 0 interceptions (statistically perfect rating 158.3) and have a better rating than Quarterback B who completes 32 out of 41 passes for 410 yards and 4 touchdowns and 1 interception (130.7). Quarterback A had a more efficient performance than Quarterback B, but which quarterback was more instrumental in the offense of their particular team? Clearly Quarterback B, but the quarterback rating does not accurately reflect that reality.
Therefore, if quarterback ratings are going to continue to be used as quantitative measurement tools in quarterback evaluation, a cap needs to be assigned to curtail the inherent exponential proficiency estimation. The best means to determine influence would be relate the cap back to yardage. The following criterion or something similar would be suitable:
Yards = Maximum Quarterback Rating Possible
0-199 = 99.9
200-249 = 119.9
250-299 = 139.9
300+ = 158.3
With the application of these caps the overall formula for calculating quarterback rating would not change, but if a quarterback failed to throw for more than 249 yards it would not matter if the formula calculated a rating of 146.7 because officially the rating would be reduced to 119.9. Moving quarterback rating beyond simple efficiency and adding game influence increases its statistical power and the ability to accurately differentiate between high quality and quarterbacks that are only asked to do so much to aid their offense, which is supposed to the real point behind quarterback rating in the first place.
Wednesday, January 27, 2010
In Search of Statistical Understanding
Mark Twain once said, “…There are three kinds of lies: lies, damned lies and statistics.” Unfortunately most people seem to have taken that statement to heart, shunning the usefulness of statistics in risk management and decision making by either not using them or not even bothering to learn proper statistics. The dearth in use of statistical information and analysis by the general public has resulted in the common misrepresentation of various pieces of information due to a lack of sufficient reported parameters. This misrepresentation has created scenarios where inappropriate decisions were favored over more rational decisions creating instances of inefficiency in already difficult situations. These scenarios are most commonly demonstrated in, but not limited to, the multitude of opinion polls that are conducted on a daily basis which supposedly dictate public policy.
Unfortunately statistical misrepresentation also infiltrates other severe issues such as medical decision-making. These misrepresentations stem either from a lack of understanding regarding how statistical theory actually operates or a deliberate attempt to boost or lower the success rate of a particular product/treatment and are perpetrated by patient, physician and pharmaceutical company. The chief concern among the public should be that inaccurate statistical analysis of these procedures at best results in a significant waste of time and money and at worst results in the greater probability of a loss of life/lives. Such a lack of statistical application is made worse by the fact that all of the relevant information is easily available, but simply not interpreted properly. Without using an objective statistical analysis how is one able to discern the difference between one procedure/product vs. another? Testimonials are rarely an appropriate determining agent due to the real possibility for a conflict of interest. Overall it is troubling that there is such a lack of importance placed on an issue that would eliminate waste at almost no additional cost and carries a high probability of saving lives.
Cancer screening and their associated false positives are a very common example of where ‘common sense’ and genuine statistical analysis part ways when coming to a conclusion regarding the result. For example the generic example used many times to illustrate this point is: if there is only a 1% chance of a women having breast cancer and a mammogram has a 90% rate of accuracy at detecting cancer in an individual that has cancer and a 9% chance of recording a false positive (the mammogram detects cancer in an individual without cancer) there is only a 9.9% chance that a positive mammogram will actually identify an individual with cancer. Such a low result is shocking to one’s natural intuition when considering only a 9% rate of false positive vs. a 90% rate of accuracy at detecting cancer, so why is 9.9% the correct result?
The basic explanation for the ‘shock’ comes from minimizing the importance of the original probability that a woman has cancer. The result is easy to understand when comparing the false positive probability rate to the actual occurrence rate. The false positive rate is nine times larger than the actual cancer occurrence rate, thus for a 100% accurate test with regards to detecting cancer in an individual with cancer there would be only a 10% chance that a positive test resulted in actually detecting cancer in an individual. In the above example the test accuracy was 90% thus there is only a 9.9% chance. Basically even with zero statistical understanding thinking about the issue properly leads one to conclusion that the correct answer needs to be somewhere in the neighborhood of the test accuracy being 10% due to the ratio between the false positive and the real positive. So in the end ‘common sense’ actually does coincide with statistical analysis as long as the ‘common sense’ used is legitimate. The sad thing is that even many physicians are surprised by this result despite the fact that they should be more in tune to such statistics.
So if there are many advantages to using statistics when making a decision, why do so many individuals elect not to use statistics? The most obvious answer is the inherent bias most individuals have towards mathematics and math related subject matter. Statistics inhabit the world of math and as a whole the part of the world they exist in is not the happy easy arithmetic neighborhood, but the difficult formula and theory neighborhood. Therefore, the application of statistics takes significant and real effort over simply punching a few numbers into a calculator; this required applied effort is another strike against statistics in a world where all things are desired to be fast and simple. The fact is that statistical analysis actually makes difficult decisions easier if used appropriately.
Another obstacle that may reduce the probability for the application of statistics in a decision-making process is a lack of certainty. Statistics do not generate a prediction of what will happen, but of what will most likely happen. Unfortunately this reality of statistics conflicts with the general psychological map that most people possess. Most individuals do not think in the context of an event happening 100 times and the probability associated with what happens each time over those 100 samples. Instead individuals focuses only on the single time that he/she will experience the particular event. This expectation leads to more trust in instinct (gut feeling) than statistics. This mindset is unfortunate because statistics exist due to the omnipresence of variability in existence including events beyond instinct.
A secondary aspect to this separation between statistics and certainty is a misunderstanding of statistics in general. Statistics generate a probability of occurrence for different possibilities over many different repetitions of the same general event. However, because a lot of event in general life do not have significant periods of repetition individuals tend to view the outcomes of those events as the actual probability of occurrence rather than what statistics predict. Basically the fact that a particular outcome only has a 3% probability of occurrence in a given scenario over 1000 tests will have little influence in the mindset of an individual that experiences that outcome 2 of the 3 times that scenarios has occurred in real life. Thus such an experience may lead an individual to doubt the accuracy and/or importance of statistics in other aspects of existence, thus leading to the incorrect viewpoint that statistics are a waste of time and effort.
In fact the power of the statistical method may also turnoff individuals because even when they choose to use statistics they can easily be disappointed by the overall power of the test because their intuition tells them the result should be more meaningful. For example suppose a brokerage firm wants to determine which of their 25 employees have been performing the most efficiently. An evaluation test is created that can identify the best performing employee with 97% accuracy. Based on statistical theory what is the actual probability that the best performing employee will be identified? Using Bayes’ theorem the evaluation test identifies the best performing employee 57.4% of the time. Although correct statistically, it does not sit well if the typical person that a test that is initially believed to have an accuracy rate of 97% in actuality only has an accuracy rate of 57.4%.
A third concern relates back to the aforementioned problem regarding available information and raises its own chicken vs. egg question. Certainly not all relevant information is going to be available to a decision-maker at the time of the decision. However, it does behoove an individual to have as much relevant information as possible regarding the issue. Unfortunately this belief does not appear to be the attitude of major polling groups and the news media as they present extremely simplified questions without any expansive circumstances. This behavior raises the question of do these polling groups behave like this because they believe that the public want simplicity and would not use additional information or do they behave like this because they are lazy and/or incompetent and cannot ask important qualifiers to their questions? This question is important because if the public learns to value statistics in decision-making then one can better assess the probability that polling groups will change their behavior when collecting and presenting information to include more details.
The reason supplemental information and qualifiers are important is because issues are rarely as simple as polling questions suggest they are. For example the most common poll question in the recent healthcare debate was ‘do you support a public option?’ Wow, what an amazingly simple question overlaying a complex issue. The first error in the question is that of the ignorant respondent. The pollster assumes that each individual answering the question has relatively the same definition for what entails a ‘public option’, which is highly unlikely. In similar fashion the pollster assumes that each individual is aware of the definition for the term being used by those in Congress. Also the pollster does not inquire to the details surrounding the success or failure of such an issue. Basically what the respondent would gain or loss if a public option existed or did not exist. None of the elements that go into creating a public option and how they would influence the answer of the respondent are discussed which defeats the point of even asking the question.
The importance of these qualifiers can be seen in the following example. Suppose you ask the following question to 1000 people: ‘Would you like 10 dollars?’ It would be very surprising if any one of the respondents answered in the negative. However, what if an important piece of information, which was excluded from the first go around, was added and another 1000 people were asked this question: ‘Would you like 10 dollars which I just stole from that 5-year old girl over there who is still being a baby and crying about it?’ Adding the information regarding the origin of the 10 dollars, another layer of complexity to the question, has changed the question dynamic completely. Now it would not be surprising if the level of response flipped to an overwhelming ‘no’. What if instead of a 5-year old girl the money was stolen from a billionaire, how would that shape the response curve?
Another problem with the media outlets and the way they diminish the importance of statistics is inappropriate presentation of growth or decline percentages. Typically this information is presented as relative changes without illustrating the absolute numbers that represent those changes (absolute changes). Not looking at the absolute changes can lead an individual to radically erroneous conclusions. For example suppose from year x to year y it is reported that the GDP in a given country increases by 25% under President A whereas 5 years ago the GDP increased by only 5% under President B. Clearly President A must be doing a better job working with Congress to manage the economy right? Not necessarily as the GDP 5 years ago could have been 3 trillion whereas in year x the GDP was 400 billion. When looking at the absolute numbers the increase in GDP 5 years ago was 150 billion whereas the increase in GDP from year x to year y is only 100 billion. So despite a 5x increase in percentage between the two equal distant time periods, the actual increase 5 years ago was 1.5x larger than the increase from year x to year y.
Reporting the absolute change is always better than the relative change because as described above, one can calculate the relative change from the absolute change, but cannot calculate the absolute change from the relative change. Unfortunately despite the above example relative changes are almost always going to be a larger number vs. absolute changes and the media in its ever expanding effort to attract more public attention over actually informing the public grab the relative change number to make the headline more important than it might actually be.
Clearly there are obstacles that need to be overcome before statistics can be implemented on a large scale. Fortunately most of these obstacles revolve around misinformation rather then difficulty of understanding. This characteristic is favorable because misinformation does not tie to intelligence, but communication and familiarity. Basically one does not need an advanced level of intelligence to understand and apply statistics.
At its heart statistics focuses on a search and discovery of patterns with later a deduction of any significant meaning to those patterns and how they may impact future events. The problem is that it tends to be difficult to perform such an exploration and analysis methodology without a proper level of experience. This lack of experience is telling in that most people are exposed to their first significant statistics course, if they are ever exposed to one in the first place, in college. It is true that the concept of probability is frequently introduced earlier than college, but in most instances such introduction does not discuss statistics and its importance in sufficient detail. College exposure is typically far too late if a goal is to develop an appreciation and understanding of statistics and what role it plays in real life. Heck, most college individuals that take statistic course lament the fact that they have to take it for their given major.
One reason for why exposure to statistics occurs at such an advanced age is that most believe that a strong core of mathematics is required before beginning study in statistics otherwise the effort applied to learn statistics will be wasted due to a lack of understanding in general mathematical theory. Unfortunately this thought process is not entirely accurate because although the study of statistics does involve advanced concepts in mathematics, there are other critical aspects to understanding the nature behind the results produced by statistical formulas.
For example one forgotten aspect of statistics is exploratory data analysis (EDA), which seeks to identify what the data is saying, not necessarily how it was calculated. EDA is an important aspect of understanding statistics because one needs to understand the context of the numbers that enter into and are spit out by statistical formulas. Also EDA focuses on using graphical information instead formula and theory which make it easier to younger students to both enjoy and understand. The application of EDA allows statistical analysts to understand why certain data should not be considered relevant for a particular statistical analysis for the inclusion of outliers or irrelevant/inappropriate data generates errors in the end result. EDA leads to the understanding of why a question like ‘what are the flaws in the methodology used for data collection’ is important to ask and how to properly answer it.
Such early analysis experience can be taught at an early level by giving students a list of data sets and details on how that data was generated and asking which sets are accurate, which sets are trash and which sets are usable as long as certain steps are taken to ensure accuracy. Also students can be asked to comment on the relevance of the outcome for certain statistical tests on given sets of data without having to do the tests themselves. Thus as a first step in renewing statistical thought in society, it would go a long way to improving the attitude individuals have towards statistics if statistical reasoning were taught before statistical theory and formulas, the mindset of statistics before the math. The issue of teaching statistics is especially pertinent to education reform. If the point of education is to ensure a populous that has the ability to reason and communicate effectively to each other in society then teaching and applying statistical reasoning is essential to achieving this goal.
Unfortunately statistical misrepresentation also infiltrates other severe issues such as medical decision-making. These misrepresentations stem either from a lack of understanding regarding how statistical theory actually operates or a deliberate attempt to boost or lower the success rate of a particular product/treatment and are perpetrated by patient, physician and pharmaceutical company. The chief concern among the public should be that inaccurate statistical analysis of these procedures at best results in a significant waste of time and money and at worst results in the greater probability of a loss of life/lives. Such a lack of statistical application is made worse by the fact that all of the relevant information is easily available, but simply not interpreted properly. Without using an objective statistical analysis how is one able to discern the difference between one procedure/product vs. another? Testimonials are rarely an appropriate determining agent due to the real possibility for a conflict of interest. Overall it is troubling that there is such a lack of importance placed on an issue that would eliminate waste at almost no additional cost and carries a high probability of saving lives.
Cancer screening and their associated false positives are a very common example of where ‘common sense’ and genuine statistical analysis part ways when coming to a conclusion regarding the result. For example the generic example used many times to illustrate this point is: if there is only a 1% chance of a women having breast cancer and a mammogram has a 90% rate of accuracy at detecting cancer in an individual that has cancer and a 9% chance of recording a false positive (the mammogram detects cancer in an individual without cancer) there is only a 9.9% chance that a positive mammogram will actually identify an individual with cancer. Such a low result is shocking to one’s natural intuition when considering only a 9% rate of false positive vs. a 90% rate of accuracy at detecting cancer, so why is 9.9% the correct result?
The basic explanation for the ‘shock’ comes from minimizing the importance of the original probability that a woman has cancer. The result is easy to understand when comparing the false positive probability rate to the actual occurrence rate. The false positive rate is nine times larger than the actual cancer occurrence rate, thus for a 100% accurate test with regards to detecting cancer in an individual with cancer there would be only a 10% chance that a positive test resulted in actually detecting cancer in an individual. In the above example the test accuracy was 90% thus there is only a 9.9% chance. Basically even with zero statistical understanding thinking about the issue properly leads one to conclusion that the correct answer needs to be somewhere in the neighborhood of the test accuracy being 10% due to the ratio between the false positive and the real positive. So in the end ‘common sense’ actually does coincide with statistical analysis as long as the ‘common sense’ used is legitimate. The sad thing is that even many physicians are surprised by this result despite the fact that they should be more in tune to such statistics.
So if there are many advantages to using statistics when making a decision, why do so many individuals elect not to use statistics? The most obvious answer is the inherent bias most individuals have towards mathematics and math related subject matter. Statistics inhabit the world of math and as a whole the part of the world they exist in is not the happy easy arithmetic neighborhood, but the difficult formula and theory neighborhood. Therefore, the application of statistics takes significant and real effort over simply punching a few numbers into a calculator; this required applied effort is another strike against statistics in a world where all things are desired to be fast and simple. The fact is that statistical analysis actually makes difficult decisions easier if used appropriately.
Another obstacle that may reduce the probability for the application of statistics in a decision-making process is a lack of certainty. Statistics do not generate a prediction of what will happen, but of what will most likely happen. Unfortunately this reality of statistics conflicts with the general psychological map that most people possess. Most individuals do not think in the context of an event happening 100 times and the probability associated with what happens each time over those 100 samples. Instead individuals focuses only on the single time that he/she will experience the particular event. This expectation leads to more trust in instinct (gut feeling) than statistics. This mindset is unfortunate because statistics exist due to the omnipresence of variability in existence including events beyond instinct.
A secondary aspect to this separation between statistics and certainty is a misunderstanding of statistics in general. Statistics generate a probability of occurrence for different possibilities over many different repetitions of the same general event. However, because a lot of event in general life do not have significant periods of repetition individuals tend to view the outcomes of those events as the actual probability of occurrence rather than what statistics predict. Basically the fact that a particular outcome only has a 3% probability of occurrence in a given scenario over 1000 tests will have little influence in the mindset of an individual that experiences that outcome 2 of the 3 times that scenarios has occurred in real life. Thus such an experience may lead an individual to doubt the accuracy and/or importance of statistics in other aspects of existence, thus leading to the incorrect viewpoint that statistics are a waste of time and effort.
In fact the power of the statistical method may also turnoff individuals because even when they choose to use statistics they can easily be disappointed by the overall power of the test because their intuition tells them the result should be more meaningful. For example suppose a brokerage firm wants to determine which of their 25 employees have been performing the most efficiently. An evaluation test is created that can identify the best performing employee with 97% accuracy. Based on statistical theory what is the actual probability that the best performing employee will be identified? Using Bayes’ theorem the evaluation test identifies the best performing employee 57.4% of the time. Although correct statistically, it does not sit well if the typical person that a test that is initially believed to have an accuracy rate of 97% in actuality only has an accuracy rate of 57.4%.
A third concern relates back to the aforementioned problem regarding available information and raises its own chicken vs. egg question. Certainly not all relevant information is going to be available to a decision-maker at the time of the decision. However, it does behoove an individual to have as much relevant information as possible regarding the issue. Unfortunately this belief does not appear to be the attitude of major polling groups and the news media as they present extremely simplified questions without any expansive circumstances. This behavior raises the question of do these polling groups behave like this because they believe that the public want simplicity and would not use additional information or do they behave like this because they are lazy and/or incompetent and cannot ask important qualifiers to their questions? This question is important because if the public learns to value statistics in decision-making then one can better assess the probability that polling groups will change their behavior when collecting and presenting information to include more details.
The reason supplemental information and qualifiers are important is because issues are rarely as simple as polling questions suggest they are. For example the most common poll question in the recent healthcare debate was ‘do you support a public option?’ Wow, what an amazingly simple question overlaying a complex issue. The first error in the question is that of the ignorant respondent. The pollster assumes that each individual answering the question has relatively the same definition for what entails a ‘public option’, which is highly unlikely. In similar fashion the pollster assumes that each individual is aware of the definition for the term being used by those in Congress. Also the pollster does not inquire to the details surrounding the success or failure of such an issue. Basically what the respondent would gain or loss if a public option existed or did not exist. None of the elements that go into creating a public option and how they would influence the answer of the respondent are discussed which defeats the point of even asking the question.
The importance of these qualifiers can be seen in the following example. Suppose you ask the following question to 1000 people: ‘Would you like 10 dollars?’ It would be very surprising if any one of the respondents answered in the negative. However, what if an important piece of information, which was excluded from the first go around, was added and another 1000 people were asked this question: ‘Would you like 10 dollars which I just stole from that 5-year old girl over there who is still being a baby and crying about it?’ Adding the information regarding the origin of the 10 dollars, another layer of complexity to the question, has changed the question dynamic completely. Now it would not be surprising if the level of response flipped to an overwhelming ‘no’. What if instead of a 5-year old girl the money was stolen from a billionaire, how would that shape the response curve?
Another problem with the media outlets and the way they diminish the importance of statistics is inappropriate presentation of growth or decline percentages. Typically this information is presented as relative changes without illustrating the absolute numbers that represent those changes (absolute changes). Not looking at the absolute changes can lead an individual to radically erroneous conclusions. For example suppose from year x to year y it is reported that the GDP in a given country increases by 25% under President A whereas 5 years ago the GDP increased by only 5% under President B. Clearly President A must be doing a better job working with Congress to manage the economy right? Not necessarily as the GDP 5 years ago could have been 3 trillion whereas in year x the GDP was 400 billion. When looking at the absolute numbers the increase in GDP 5 years ago was 150 billion whereas the increase in GDP from year x to year y is only 100 billion. So despite a 5x increase in percentage between the two equal distant time periods, the actual increase 5 years ago was 1.5x larger than the increase from year x to year y.
Reporting the absolute change is always better than the relative change because as described above, one can calculate the relative change from the absolute change, but cannot calculate the absolute change from the relative change. Unfortunately despite the above example relative changes are almost always going to be a larger number vs. absolute changes and the media in its ever expanding effort to attract more public attention over actually informing the public grab the relative change number to make the headline more important than it might actually be.
Clearly there are obstacles that need to be overcome before statistics can be implemented on a large scale. Fortunately most of these obstacles revolve around misinformation rather then difficulty of understanding. This characteristic is favorable because misinformation does not tie to intelligence, but communication and familiarity. Basically one does not need an advanced level of intelligence to understand and apply statistics.
At its heart statistics focuses on a search and discovery of patterns with later a deduction of any significant meaning to those patterns and how they may impact future events. The problem is that it tends to be difficult to perform such an exploration and analysis methodology without a proper level of experience. This lack of experience is telling in that most people are exposed to their first significant statistics course, if they are ever exposed to one in the first place, in college. It is true that the concept of probability is frequently introduced earlier than college, but in most instances such introduction does not discuss statistics and its importance in sufficient detail. College exposure is typically far too late if a goal is to develop an appreciation and understanding of statistics and what role it plays in real life. Heck, most college individuals that take statistic course lament the fact that they have to take it for their given major.
One reason for why exposure to statistics occurs at such an advanced age is that most believe that a strong core of mathematics is required before beginning study in statistics otherwise the effort applied to learn statistics will be wasted due to a lack of understanding in general mathematical theory. Unfortunately this thought process is not entirely accurate because although the study of statistics does involve advanced concepts in mathematics, there are other critical aspects to understanding the nature behind the results produced by statistical formulas.
For example one forgotten aspect of statistics is exploratory data analysis (EDA), which seeks to identify what the data is saying, not necessarily how it was calculated. EDA is an important aspect of understanding statistics because one needs to understand the context of the numbers that enter into and are spit out by statistical formulas. Also EDA focuses on using graphical information instead formula and theory which make it easier to younger students to both enjoy and understand. The application of EDA allows statistical analysts to understand why certain data should not be considered relevant for a particular statistical analysis for the inclusion of outliers or irrelevant/inappropriate data generates errors in the end result. EDA leads to the understanding of why a question like ‘what are the flaws in the methodology used for data collection’ is important to ask and how to properly answer it.
Such early analysis experience can be taught at an early level by giving students a list of data sets and details on how that data was generated and asking which sets are accurate, which sets are trash and which sets are usable as long as certain steps are taken to ensure accuracy. Also students can be asked to comment on the relevance of the outcome for certain statistical tests on given sets of data without having to do the tests themselves. Thus as a first step in renewing statistical thought in society, it would go a long way to improving the attitude individuals have towards statistics if statistical reasoning were taught before statistical theory and formulas, the mindset of statistics before the math. The issue of teaching statistics is especially pertinent to education reform. If the point of education is to ensure a populous that has the ability to reason and communicate effectively to each other in society then teaching and applying statistical reasoning is essential to achieving this goal.
Subscribe to:
Posts (Atom)