The Austin Animal Center, the largest no-kill municipal shelter in the United States, makes available its collected data on Austin's Open Data Portal. This data includes both animals incoming into the shelter and the animals' outcome. In this post, we perform some exploratory data analysis on the intakes dataset to see if we can find any noticeable trends or interesting pieces of information of the data. First, we will extract the data from Austin's Data Portal, which is supported by Socrata. We will then perform some data transformation and cleaning steps to get the data ready for analysis.

Getting Started¶

Before diving in, import the libraries that we will use throughout the analysis. %matplotlib inline is used for displaying the visualizations.

Extracting the Animal Intakes Dataset¶

The Socrata Open Data Access (SODA) API is used to extract the data from the Open Data Portal. The Socrata API documentation states there is a call limit to minimize the impact on their servers. Therefore, we need to page the results to collect the entire dataset. The following function extracts the data from the API and transforms the default JSON result format into a pandas DataFrame when the return_df parameter is set to True.

The intakes dataset API endpoint can be found in the upper-right corner of the dataset's information page by clicking the API button and copying the URL endpoint. The data can also be downloaded as a CSV by changing the JSON output to CSV.

With the endpoint in hand, we can use our new function to get the intakes dataset! When the data has been paged through and collected into a DataFrame, we print the number of rows and the first five rows of the dataset using head() to make sure we successfully extracted the data.

The dataset contains just over 80k rows and twelve columns, which matches the information on the dataset's open data portal page. The column names all match what is listed on the information page as well. Therefore we have successfully extracted the data programmatically! Now that we have the data in a tidy pandas DataFrame let's save it to a CSV using pandas' convenient to_csv() IO method, so we don't have to call and get the data from the Socrata API repeatedly.

With the data extracted and saved so we can access it later if needed, we can move to transforming and cleaning the data!

Feature Engineering and Data Preparation¶

Although the dataset we have now is tidy already, there are a few more steps we can take to make the data more useful in what is collectively referred to as feature engineering. The process of feature engineering can often be a manual task, but the addition of more data points given the information already available can be a powerful way of enhancing a dataset's information. We know from the dataset's information that each row represents an individual intake record. Thus we can also add a count column for convenience.

The ages of the animals entering the shelter are given by a numeric value and unit of time, for example, '1 year', '3 months', or '18 weeks'. In most cases, estimated age is the best available due to the circumstance of the shelter environment, but the current format is not very suitable for analysis. Therefore, we convert the approximated age given by the age_upon_intake column into the number of days which allows us to get a numeric value of the animals' age that can be transformed into a consistent unit of time, such as years. The datetime column is also transformed to a usable datetime format using pd.to_datetime(). The following are some simple functions that take the DataFrame and the respective age and date columns as arguments.

Use the above functions to transform the date and age columns and add some more informative columns that will be used for exploring the data.

Print the resulting shape of the new DataFrame with the additional columns and the first five rows.

Exploratory Data Analysis of the Animal Intakes Dataset¶

Before beginning, we quickly set the overall style of the plots that will be created throughout the analysis.

Now that the extracted raw dataset has been transformed with additional features and information, we can begin exploring the data! First, let's get a sense of the proportion of different intake conditions and types, as well as the age and type of the animal entering the shelter.

We see the majority of animals entering the shelter are strays without any injuries or illness and approximately within 2.5 years of age. Owner surrenders are the next most common intake type, which can occur for a variety of reasons such as a significant life change or a death in the family but is entirely behind strays. Dogs and cats are by far the most common animals to be brought to the shelter, perhaps to no one's surprise. What is also interesting is the vast disparity between the age of the animal entering the shelter. As dogs and cats have approximately the same average lifespan, would we have expected to see a more even distribution of ages? Or, are younger dogs and cats more likely to be picked up as strays? One could likely reason those younger animals would make up a higher proportion of strays, especially for cats.

Knowing that the overwhelming majority of animals entering the shelter are relatively healthy strays within 2.5 years of age and are either a dog or cat, is it likely most of these animals are intact (not neutered or spayed)? As cat overpopulation (especially in perpetually warm and temperate climates like Austin and the surrounding area) is likely a concern and trap neuter release programs become more common, would we expect to see more intact cats entering the shelter than dogs?

Intact dogs enter the shelter at a slightly higher volume than cats; however, notice neutered cats enter the shelter at a much lower amount than dogs, which could indicate that when a cat does enter the shelter, it is more likely to be intact then spayed or neutered, which fits into our previous assumptions.

Are there any significant differences in the number of animals intaken at particular periods of time, such as the day of week or month of the year?

There isn't much difference in volume depending on the day of the week, with perhaps the exception of Sunday. May through June and October through November are the busiest times of the year based on overall animal intake volume. Cats most commonly begin having litters in early spring, peaking in late spring and ending in the fall, according to the Humane Society of America which roughly aligns to the most active months of the year. Dogs do not have a 'breeding season' per se. Therefore it may be more difficult to determine if months with more intakes are caused by dogs giving birth to litters of puppies.

We see a much more substantial distinction between the volume of intakes at the hourly level, where a large proportion of intakes occur from 11 am - 2 pm. This volume likely coincides with the shelter's opening hours as the amount tapers throughout the day and we see very few intakes made after 8 pm.

As we have about four years' worth of data, let's plot the number and types of animal intakes over time to see if there are any noticeable trends or other points of interest. We noted stray animal intakes are by far the most common intake. Therefore, an additional plot is created without strays to get a better sense of the trend for the other intake types.

It seems stray animal intakes peak in the first half of the year around April-May and then steadily tapers off until the next year. This result appears interesting as one could reasonably assume that stray animal intakes would be consistent throughout the year as pets go missing and strays are picked up through various trap-neuter-release and other programs. The peaks could be a result of animals giving birth and thereby increasing the number of strays that would be picked up. This result would align when 'cat breeding season' (though there isn't technically such a term), starts to pick up in the spring, as we noted earlier. There is a slight downward trend to the number of strays taken in each year, which could be construed as positive (more resources and efficient programs, for example), or negative (lack of resources means more strays are still on the street and not taken to the shelter). 2018 appears to be a meager year regarding stray intakes, hopefully for the former reason.

Although much less significant in overall volume compared to strays, wildlife intakes appear to have a similar trend to stray intakes, peaking in the first months of the year and then steadily decreasing. Owner surrender intakes dip and rise rather frequently, perhaps there are certain parts of the month when owner surrenders are more common. Public assistance intakes look to be the most consistent regarding volume; however, there is an interesting spike in the latter half of 2017.

Earlier, we noted there wasn't much difference in the number of intakes by day of the week, but does this hold for owner surrenders? Given surrenders are from people outside the shelter organization, would we see more surrenders on weekends?

The days with the most owner surrender intakes are rather similar to the overall volume of intakes during the week and do not differ much from each other. Given that people with all types of schedules own pets, these results are not surprising, though our original guess of more owner surrenders during the weekend turned out to be a false assumption. The owner surrender intake volume would also be bounded by the operating hours of the shelter.

Investigating further into the trend of owner surrenders, let's plot only surrenders from the beginning of 2017 by month.

Peaks appear to occur in January, February, May, August, September, and December. What could be some causes of this trend? Knowing that owner surrenders increase in the winter months January, February, and December, one possible reason that comes to mind is owners surrendering animals received as gifts over the holidays (a terrible idea, by the way!). August and September are about when cat breeding typically decreases. Thus, one explanation could be that kittens that were adopted at the beginning of kitten season in early April would be adults at that point. If not appropriately trained, kittens can develop several behavioral issues into adulthood. Perhaps the owners who adopted kittens did not want the cat after it became an adult? A depressing hypothesis, to be sure.

Let's quickly check to see if there's any significant difference in the number of intakes at the beginning and end of the month compared to the rest of the month. We use pandas's datetime functions dt.is_month_start and dt.is_month_end and find the average of the number of intakes.

Not much of a difference on average. We had no prior reasoning to assume there would be a difference (although if we were analyzing bank deposit data over time or similar that assumption would be reasonable), so these results aren't surprising.

So far, we've only explored the intakes into the shelter in aggregate. Let's see if any animals have been taken into the shelter multiple times by applying the value_counts() method to the animal_id column and plotting the first fifteen values.

Some animals have been intaken into the shelter over ten times! The remaining top fifteen most frequently intaken animals hover around 6-8 intakes. Are animals often taken into the shelter more than once? Let's get the overall count of the frequency of intaken animals with the same unique id.

Construct a quick DataFrame to display the frequency of multiple intakes.

About 6,100 animals out of the 80,000 records or about 7.63% in the dataset have been taken into the shelter more than once. This seems like a relatively high percentage given the overall number of records and intakes performed by the shelter. However, we don't have any comparison data to analyze if the number of multiple intakes is high, so for all we know this percentage could also be comparatively low.

Let's dig a little deeper into the subset of animals that have been intaken multiple times. We filter the intakes dataset for animal ids that appear more than once.

Using the filtered dataset, let's plot the animal and intake type to see if there are any significant differences.

Stray dogs and, to a lesser extent, owner surrendered dogs, appear to represent the majority of animals intaken multiple times. Stray and owner-surrendered cats are about even regarding overall count but still are dwarfed by dogs intaken numerous times. We saw earlier that stray animal intakes were the most common type of intake received by the shelter, so it is not surprising to see stray intakes as a significant proportion of multiple intakes. However, this result seems to imply dogs that are released from the shelter have a higher chance to go stray or be surrendered by their owners compared to cats. As dogs are typically outdoors more often and therefore have a higher chance to go missing, as well as having a higher level of care required compared to cats (in general), perhaps this result isn't beyond a reasonable doubt.

To identify animals that have been taken in multiple times and create a unique key, we add an intake_number column using rank() on the datetime of the intake. We then concatenate the original animal_id column with the newly created intake_number column to create a unique identifier column that will be useful in later analysis.

Conclusion¶

Through our exploratory analysis of the Austin Animal Center intakes dataset, we were able to reveal several exciting pieces of information about the animals intaken into the shelter. To briefly summarize some of what we uncovered:

• The overwhelming majority of intakes are stray dogs and cats within 2.5 years of age and in relatively healthy condition. Dogs are taken in more frequently than cats, which is interesting as one might reasonably expect there to be a more even split between dog and cat intakes. Particularly in a temperate and warm climate like Austin, where cats would be able to breed more often and more kittens surviving into adulthood. However, cats are much more difficult to capture than dogs (some live traps in use in trap-neuter-release programs can be set for days or weeks before a cat is caught), due to their size and ability to hide in smaller and more hidden spaces.
• Most cats and dogs taken to the shelter are not spayed or neutered, which given that the majority of intakes are strays, is not a surprise. Interestingly, the difference between spayed/neutered cats and dogs is much more prominent than intact intakes. Thus, it is much more likely that a cat intaken into the shelter will not be spayed or neutered. According to the Humane Society of America, approximately 2 percent of community cats are spayed or neutered. Given such a low proportion of spayed and neutered cats living outdoors, it is not unexpected to see comparatively low spayed/neutered cat intakes.
• There is not much difference in the volume of intakes over the course of a given week, though we do see overall intakes increase in May, June, and November. 'Kitten season' is typically in full swing by May and June, which could partially explain the increase. Most intakes are done between 11 am, and 4 pm, which is reasonable as intakes would be constrained by the hours the shelter is open.
• Stray and, to a lesser extent, wildlife intakes appear to peak in the beginning months of the year and then steadily decrease throughout the remainder of the year. Owner surrenders are relatively consistent in comparison, with some minor peaks and troughs in certain months.
• Dogs are more likely to return to the shelter either as a stray or as an owner surrender. Almost 8% of the animals intaken into the shelter were previously intaken.

In the next analysis, we combine the intakes and outcomes datasets provided by the Austin Animal Shelter to explore further into trends of what occurs during an animal's time in the shelter.

Lastly, save a copy of the dataset with the additional features we created during the analysis. We first delete the intermediate and no longer needed columns that were created during the analysis.