Summary of 2023 StatStart Program

Tue, 25 Jul 2023 21:13:14 -0500

Introduction

Hello to everyone reading this post. The objective for this post is to showcase some of the numerous things me and my fellow stat start program people have done throughout the course of this 4 week program. In this blog post you will see some graphs, histograms, data (obviously), and many more to showcase just how much we have grown within these very short 4 weeks.

The format in this post will be as followed:

Data Analysis Portion of stat start
- Will show bits of code from R
Florence Nightingale Data
- City ownership
- Stat of life
Statistical analyzation
- Misconceptions
- Compare and contrast trends

(For those who want more details on how to use R Markdown check out this link http://rmarkdown.rstudio.com)

Data Analysis Portion

(Part by: Ryan Mac) (Note: The following data set used is provided by the DSLabs library)

An example data set that we’re going to examine in this post is the 2010 NYC Regents Exam data collection provided from the DSLabs library. To keep the length of the blog relatively sized, we’re only going to take a look at the statistical summary within the given data set.

For better context to this data set, the data set is split into 6 data categories ranging from the student’s overall score to the students explicit subject scores like Integrated Algebra, US History, or English. With that in mind, lets take a look at what happens when we plot the mean of this data set:

# For this visualization, we're going to need DSLabs to get the NYC Regents data set and ggplot 2 for graphing 

library(dslabs)
library(ggplot2)
library(tidyverse)

# Now let's actually graph these data parts

# Mean of Overall Scores
ggplot(data=nyc_regents_scores,aes(x=score))+
 geom_histogram(aes(y =..density..),binwidth=1000,color="black",fill="white")+
  ggtitle("Overall Student Scores")+
  xlab("Score Range")+
  ylab("Density (# of People)")+
  theme_bw()+
  geom_vline(xintercept=mean(nyc_regents_scores$score,na.rm=T),color="blue",linetype="dashed")


# Mean of Integrated Algebra Scores
ggplot(data=nyc_regents_scores,aes(x=integrated_algebra))+
 geom_histogram(aes(y =..density..),binwidth=1000,color="black",fill="white")+
  ggtitle("Integrated Algebra Scores")+
  xlab("Score Range")+
  ylab("Density (# of People)")+
  theme_bw()+
  geom_vline(xintercept=mean(nyc_regents_scores$integrated_algebra,na.rm=T),color="blue",linetype="dashed")


# Mean of Integrated Algebra Scores
ggplot(data=nyc_regents_scores,aes(x=english))+
 geom_histogram(aes(y =..density..),binwidth=1000,color="black",fill="white")+
  ggtitle("Integrated Algebra Scores")+
  xlab("Score Range")+
  ylab("Density (# of People)")+
  theme_bw()+
  geom_vline(xintercept=mean(nyc_regents_scores$english,na.rm=T),color="blue",linetype="dashed")

At first glance, the overall score plot seems weird mainly because it seems like everyone had an equal distribution of scores. However, in subjects like Integrated Algebra or English, it seems as if the majority of people failed the subject! Since there’s a majority of failure, plotting the mean in this case results in an unrepresentative average report since the data is skewed. If we were to plot something like the median, we should see a central point that makes more sense:

# Median of Overall Scores
ggplot(data=nyc_regents_scores,aes(x=score))+
 geom_histogram(aes(y =..density..),binwidth=1000,color="black",fill="white")+
  ggtitle("Overall Student Scores")+
  xlab("Score Range")+
  ylab("Density (# of People)")+
  theme_bw()+
  geom_vline(xintercept=median(nyc_regents_scores$score,na.rm=T),color="blue",linetype="dashed")


# Median of Integrated Algebra Scores
ggplot(data=nyc_regents_scores,aes(x=integrated_algebra))+
 geom_histogram(aes(y =..density..),binwidth=1000,color="black",fill="white")+
  ggtitle("Integrated Algebra Scores")+
  xlab("Score Range")+
  ylab("Density (# of People)")+
  theme_bw()+
  geom_vline(xintercept=median(nyc_regents_scores$integrated_algebra,na.rm=T),color="blue",linetype="dashed")


# Median of Integrated Algebra Scores
ggplot(data=nyc_regents_scores,aes(x=english))+
 geom_histogram(aes(y =..density..),binwidth=1000,color="black",fill="white")+
  ggtitle("Integrated Algebra Scores")+
  xlab("Score Range")+
  ylab("Density (# of People)")+
  theme_bw()+
  geom_vline(xintercept=median(nyc_regents_scores$english,na.rm=T),color="blue",linetype="dashed")

In this case, the plot driven by R code seems to work properly as it should. As for why the data looks strange, we’ll cover that later on in this blog. For now, let’s take a look at another example of doing data analysis in R.

Florence Nightingale Data Portion

(Part by: Joshua Hennes) (Credits to Professor Kelly and her team for creating this dataset used within this part of the blog)

In this section of the post we will be looking at the data comparisons within different trees in Cambridge and Boston. The objective we tried to do using this dataset was to figure out how to use R to showcase the different ownership of trees and the

So let’s get started. The first thing we needed to do was to download and install tidyverse. This is to help with the creation of our graph. Once tidyverse was installed we ran this line of code below:

library(tidyverse)

The reason we ran library(tidyverse) instead of just using tidyverse is because in order for R to fully understand that we specifically want to use the functions within tidyverse, we have to call it. Henceforth why we use the library() which is designed to tell R studio what we want to call and use functions of.

After that was finished, we then had to get the dataset which contained the tree data. So we ran these two codes below:

# this chunk is to install the package
install.packages("devtools")
devtools::install_github("harvard-ufds/bosTrees")

library(bosTrees)

# This is to load the data
data(bosTrees)
data(camTrees)

Now here is the fun part. So as stated above, our first objective is to showcase the different ownership within the streets of Cambridge. So now to do this we wrote these lines of code which I will explain just after this:

ggplot(data = camTrees, mapping = aes(x = Longitude,
                                      y = Latitude,
                                      color = Ownership)) +
  geom_point(alpha = 0.05) +
  guides(colour = guide_legend(override.aes = list(alpha = 1)))

Let’s dissect this code line by line. So the reason why we use ggplot() is because it’s the function to help us create the graph we want to portray. But if we just said ggplot(), it would give us a blank graph with nothing on it. Henceforth why we give it parameters.

The first of these parameters being which dataset we want to use within our saved Data. To do this we said data = camTrees, to let R studio know we want that specific dataset.

After that, we use the mapping function so as to let R Studio know just how to plot these different points on the graph (Which is why when plotted they make the various streets of cambridge). In which we set in the aesthetic: the X to be longitude, and the Y to be Latitude. Once all of that is done, we set the colors to be equal to ownership. This way we can tell who owns which trees. But we are not done yet, since we have two more lines to talk about.

Let’s start with the guides() line first. Why this line is extremely important is because we need a type of legend to indicate to us just what these colors mean. Cause let’s say we never created that line. Well you will indeed get a graph that shows different colors, but we still wouldn’t know who owns what because there would be no legend to tell us just what the colors stand for.

Now somebody may ask the question “Well that makes sense. But what does the override.aes=list(Alpha = 1) mean?” Well I will explain both that and what the line above stands for. Now the reason why we said the override function, is because the line above the guide function is a transparency function. Which counts for anything relating to the different ownership colors. So if we were to just run the entire code without having the override in there, we would indeed have a legend, but the colors would be so hard to see. So that’s why we added the override function to keep the color non transparent and easier to see.

With all of that being said, congratulations! You just created your first Graph and learned a bit more about R Studio!. However, I would implore you to mess around with the different variables you can add into the graph. For example, instead of ownership, why not switch it for another variable within the dataset. Like the Solar variable (Wink, wink, nudge, nudge).

Statistical Analysis

So, we’ve come to this point where we’ve shown plotted data sets using R code and as seen in the post, it’s working as it should. Plots in R code however are only half of the story, now it’s time to interpret this whole thing.

Let’s start off with the 2010 NYC Reagents data. Looking back at the original graphs, you may ask “Why do your graphs look very weird and skewed?”. The answer to that is worth examining the data set itself.

(Show image of improper data set)

What you’re looking at here is a data row where in this case a student got a perfect score of 100. However, looking past that category, all the other individual subject scores appear as “NA” data, and this is just one out of many scenarios that appear like this. Situations like this can easily cause misinterpretation of the data because although the student got a perfect score on the exam overall, none of their subject data recorded, which may contribute to “Skewed data plots” shown in subjects like Integrated Algebra or English. Knowing the errors within the data set, this misinterpretation raises questions like “In reality, how many people got a perfect score?” and that question can be very difficult to solve.

Misconceptions can also arise in the Boston Trees data set as well. Although there aren’t as many marginal errors shown in other sets like the 2010 NYC Regents Exam, sometimes datasets may not be up to date to new data. For example, the data set can contain data with a date for when a tree date listed and no removal date, but when searching through maps (or visiting the tree’s location in person), the tree may not exist in that spot anymore which can arise such questions like “Was the tree planted even real?” or “When did the tree actually get removed?”, going back to the point that these misconceptions may make solving data questions a mystery like Big Foot.

With that being said, you’ve made it to the end of this post! Hopefully you’ve found this blog post to be intriguing or informative. (Fun Fact: This whole post is made using R too!). Since this is our first written by the two of us (Ryan & Joshua), we’ll be making more posts to our own respective blog websites so be sure to follow up with that as our blog sites progresses!

Hello R Markdown

Tue, 01 Dec 2020 21:13:14 -0500

R Markdown

This is an R Markdown document. Markdown is a simple formatting syntax for authoring HTML, PDF, and MS Word documents. For more details on using R Markdown see http://rmarkdown.rstudio.com.

You can embed an R code chunk like this:

summary(cars)
##      speed           dist       
##  Min.   : 4.0   Min.   :  2.00  
##  1st Qu.:12.0   1st Qu.: 26.00  
##  Median :15.0   Median : 36.00  
##  Mean   :15.4   Mean   : 42.98  
##  3rd Qu.:19.0   3rd Qu.: 56.00  
##  Max.   :25.0   Max.   :120.00
fit <- lm(dist ~ speed, data = cars)
fit
## 
## Call:
## lm(formula = dist ~ speed, data = cars)
## 
## Coefficients:
## (Intercept)        speed  
##     -17.579        3.932

Including Plots

You can also embed plots. See Figure 1 for example:

par(mar = c(0, 1, 0, 1))
pie(
  c(280, 60, 20),
  c('Sky', 'Sunny side of pyramid', 'Shady side of pyramid'),
  col = c('#0292D8', '#F7EA39', '#C4B632'),
  init.angle = -50, border = NA
)

Figure 1: A fancy pie chart.

R | Hennes Archives