The Austin Animal Center provides its animal intake and outcome datasets on Socrata. When an animal is taken into the shelter, it is given a unique identifier that is also used in the outcomes dataset. We have already investigated and performed exploratory data analysis on the Austin Animal Center's intakes and animal outcomes individually and found several interesting facets of information. In this analysis, we merge the intakes and outcomes dataset using pandas to enable us to perform exploratory data analysis on the merged data. With the data merged, we will be able to explore in more depth the transition from intake to outcome.

Getting Started¶

As before, we import the libraries that will be used throughout the analysis. The %matplotlib inline is used for displaying the visualizations.

The following imports a Python module containing the get_soda_api_data() function and the other transformation function used in the intakes analysis. The code can be found as a Github Gist.

Getting the Data¶

As seen in previous analyses, the Austin Animal Center datasets are hosted on the Socrata Open Data Access (SODA) API. We've already extracted the intakes dataset in the most recent analysis. Therefore, we only need to extract the outcomes data using the SODA API. We extract the AAC outcomes dataset using the get_soda_api_data() function from the aac_analysis module.

Print the dimensions of the extracted data and the first five rows.

There are slightly more than 80k rows and 12 columns, which match the metadata available on the outcomes data page on Socrata (at the time of this writing). Now that we have the data extracted from the SODA API and converted into a pandas DataFrame let's save the data as a CSV file to avoid the need to call the Socrata API more times than necessary.

Creating and preparing the merged dataset¶

Now that we have the outcomes dataset, we begin adding more features to the data similarly to the intakes. As we have already created the necessary functions during the intakes analysis, this step is merely a matter of applying the functions to the data. Before that, however, we create another function for adding more features related to the animal's estimated date of birth, which is provided in the outcomes data but not the intakes.

We now add more features to the outcomes using our previously developed functions and the newly written transform_birthdates() function.

The additional feature columns are now added to the outcomes data! The last step in creating the dataset is to load the intakes data we previously worked with and merge the two sets. We load the intakes data that was saved at the end of the analysis with all the added features already included using pandas's read_csv.

Both datasets are now loaded as DataFrames. Before proceeding to merge the data, we take an additional step of dropping the duplicate columns in both datasets. First, let's check to see what and how many duplicate columns are in both datasets.

We want to keep some of the columns like breed and color in one dataset, but drop the duplicate and no longer needed columns in the other dataset. We choose to keep the still relevant columns in the outcomes dataset, not for any particular reason as the values should be the same across the data. One could investigate further and see if the animal's descriptive values like color change from intake to outcome are indeed true, but this isn't germane to our particular analysis.

The unneeded and duplicate columns are now removed from the datasets. Therefore, we can proceed to merge the data! The unique identifier we created using the provided animal's id and intake and outcome number is set as the index for both datasets. This unique id allows us to perform the merge appropriately as some of the animal IDs in both datasets are not unique, due to some animals being taken into the shelter on multiple occasions. The merge itself is performed using pandas's merge function, which merges two DataFrames by using a SQL-style join.

Let's check the columns of the new dataset along with the dimensions and the first few rows of the data.

With one merged dataset that contains the available data of an animal's intake to outcome, we can find the amount of time the animal was in the shelter before its outcome. The intake and outcome datetimes are converted into datetime objects using pandas's to_datetime() function. We can then subtract the times to create a time_in_shelter column.

Using the describe() method, we can get a handy table of descriptive statistics that gives us a general sense of the distribution of the newly created time_in_shelter column.

Interestingly, the minimum time an animal was in the shelter is $-1,607$ days, which is obviously not a real value. Having a negative number of days in the shelter would imply a number of the intake datetimes are more recent than their respective outcome datetimes. To investigate a little further, we can filter the dataset for intake datetimes that are more recent than their outcomes.

The length of the filtered data indicates there are just over 500 records that have a more recent intake than outcome. As there are just under 80,000 rows in the entire dataset, it is likely the incorrect datetimes are merely a matter of the data being entered incorrectly. Assuming the intake and outcome dates are switched, we can correct the datetimes by replacing the values when the intake is more recent than the outcome.

We can now update the time_in_shelter column using the corrected dates and print the descriptive statistics table to see if the distribution is more in-line with reality.

Much better! We see the mean number of days from intake to outcome is just over two weeks, and about 50% of the animals incoming to the shelter will remain there for less than five days. The most prolonged stay in the shelter is 1,606 days or just under 4.5 years. To double-check that the intake and outcome dates are adjusted, we can filter the dataset again as before. If there are no intake dates that are more recent than the outcome, the returned DataFrame should be empty.

The filtered DataFrame is indeed empty! Now that we have merged the intakes and outcomes datasets and cleaned and transformed the data, we can proceed to perform exploratory data analysis! Before moving to the next step, let's quickly remove some of the no longer needed columns left over from the merge and save the data to a CSV.

Analyzing animals' journey from intake to outcome¶

Now we're in the heat of the meat! The exploratory data analysis we perform here will be instrumental in later steps when constructing prediction models as we will already have an understanding of what variables are seemingly crucial in determining the animals' length of stay at the shelter or outcome type. Before diving in, we quickly set some default styles for the plots that will be created with seaborn using set.

To start, let's get a sense of the distribution of animal types that are included in the data using value_counts.

As one might have reasonably assumed, the majority of animals taken into the shelter are cats and dogs. Unfortunately, there is no information regarding what entails the Other category, while birds and livestock are taken into the shelter rather rarely in comparison. There are only nine livestock records, which is rather low for our analysis and also not as much of interest. Therefore we filter out these records with .loc.

Let's investigate the mean duration an animal stays at the shelter until its outcome based on the animal's age group and type. Before plotting, we create a new column that is the float representation of the time_in_shelter column, which has a timedelta data type. This conversion can be done by accessing the total_seconds() method of the dt attribute and dividing it by the number of seconds in one day.

Cats stay in the shelter longer than any other animal. The length of stay also increases for cats as their age increases, up until about 12.5 years old. Interestingly, the time in the shelter for dogs appears to be relatively flat regardless of the age of the dog. Anecdotally, I have seen cats of a higher age remain at local shelters longer on average in general, so these results aren't too surprising in that regard. It is unfortunate, however; since cats are prolific breeders, and in an environment such as Austin, likely breed year round, there is a high chance kittens are coming into the shelter at a much higher rate than cats of an older age. As kittens are adopted much quicker than adult cats in general, older cats can end up staying in a shelter environment longer than younger cats. The data seems to confirm this, as the majority of the cat intakes (and intakes overall) are within 2.5 years of age.

Through the combination of the intake and outcome types, we can see the count of overall intakes and their resulting outcomes. As we've seen in the separate intake and outcome analyses, stray animal intakes and adoption or transfer outcomes represent the vast majority of the cases. Therefore, a combination of these intakes and outcomes will likely be the majority of overall cases. The following barplot depicts the top 10 intake to outcome cases concerning total count.

As we suspected, stray intakes to adoption or transfer outcome are the majority of cases in the data, followed by lost animals being returned to their owners (Stray -> Return to Owner) and pets being surrendered by their owners and later adopted (Owner Surrender -> Adoption). Judging from the above plot, it appears the first few intakes to outcome paths represent the majority of overall cases, but we can quickly find the cumulative proportion of the intake to outcome paths with a quick and dirty loop.

The reported cumulative proportions show the first ten intake to outcome paths regarding overall volume represent about 95% of all cases. Stray intakes to adoptions or transfers alone make up just over half of all animal records. Now that we know more about the volume and types of intakes to outcomes that appear at the shelter, we can turn our attention to finding out which intakes stay in the shelter the longest and the common outcome. The following groups the intakes and outcome results by the mean number of days in the shelter.

Missing outcome cases and euthanasia requests to adoptions appear to take the longest to resolve on average. Stray and wildlife intakes, as well as owner surrenders that end up in adoptions, take approximately one month, or about 30 days. As shelters are often short on space for animals and resources in general, knowing the average amount of time an animal may be in the shelter before its outcome can be valuable information for keeping overflow under control.

In the outcomes analysis, we noted that an intact animal (not spayed or neutered) would not be adopted before being altered to help keep the animal population down. It was assumed that transfer outcomes represented intact animal intakes being transferred to a different facility to be spayed or neutered. Now that we have the intake sex and outcome sex data resulting from the combination of the two datasets, we can see if this assumption was accurate. The following plots the top 15 intake to outcome cases concerning overall volume, which we saw earlier makes up over 95% of all cases, by the animal's intact status upon intake and outcome.

First, it appears there are a few records that may be mislabeled, as the intake status is altered, but the outcome is intact, which would imply the spay or neuter operation was somehow reversed. Fortunately, however, this mislabeling appears only in four cases. As we can see from the Intact -> Altered bar plot, by far the most common intake to outcome is a stray intake to adoption, which shows that the animal is spayed or neutered without being transferred. This result indicates that our original assumption of intact animals being transported to partner facilities before being adopted was incorrect and that spays or neuters do occur on-site before adoption. The majority of altered animal intake to outcomes are either owners surrendering their pets to the shelter or the shelter returning a missing pet to its owner, followed by already altered strays being adopted.

Feral and nursing animals are mostly transferred to other facilities, while pregnant animals are likely to be adopted or transferred. The shelter would likely want to adopt a pregnant pet to an owner who is willing to take care of the resulting offspring rather than further burden any of the shelter's partner facilities. About 50% of sick animals entering the shelter needed to be euthanized, however, the remaining 50% are either transferred or adopted. It is possible the animal will still need to be euthanized after being moved, but hopefully, the partner facility will have the resources to administer more effective care. Injured animals are also often euthanized, suggesting the injury is severe, and euthanasia was, unfortunately, the best course of action. However, injured animals are also often adopted or returned to their owner at a seemingly higher proportion than sick animals, implying the animal has or will recover from its injury.

Does the condition of the animal upon intake help determine how long an animal may need to stay at the shelter? One would assume that sick or injured animals may need to stay longer to convalesce and that older animals may remain in the shelter for more days on average before being adopted.

Most non-normal intake conditions that eventually lead to an adoption appear to take the most days on average, ranging anywhere from about 50 to 60 days. Animals that go missing from the shelter also seem to take the longest to resolve on average. Although it's unclear how an animal would go missing from the shelter, the length of time between the intake and the missing outcome is likely the amount of time the staff keeps a search for the animal before it receives the 'Missing' outcome categorization.

Conclusion¶

Through the combination of the Austin Animal Center's intakes and outcomes datasets and some cleaning and feature engineering of the data, we were able to investigate further into the complete process from the animal's intake into its outcome. By performing exploratory data analysis on the merged and transformed dataset, we revealed several exciting pieces of information that gives us more knowledge of what to expect when an animal arrives at the shelter regarding the time of stay and possible outcomes. This information will be of critical value when building and validating prediction models as we have prior knowledge of the factors and other components of the data that appear to have importance in predicting the animal's length of stay in the shelter or its outcome.