Assignment 7 - Pandas

Due Oct 19.

In this assignment we will use pandas to examine earthquake data.

Start by importing pandas, numpy and matplotlib.

I saved you some time by pre-downloading some data in .csv format from the USGS Earthquakes Database. It is located at:

You don't even need to download it. You can open it directly with Pandas.

1) Use Pandas' read_csv function directly on this url to open it as a DataFrame

(Don't use any special options). Display the first few rows and the DataFrame info.

You should have seen that the dates were not automatically parsed into datetime types.

2) Re-read the data in such a way that all date columns are identified as dates and the earthquake id is used as the index

Verify that this worked using the head and info functions.

3) Use describe to get the basic statistics of all the columns

Note the highest and lowest magnitude of earthquakes in the databse.

4) Use sort_values to get the top 20 earthquakes by magnitude

Examine the structure of the place column. The country information seems to be in there. How would you get it out?

5) Extract the country using Pandas text data functions

Add it as a new column to the dataframe. (Is it really just country? No, some rows have the name of a US state.)

6) Find the 10 states / countries with the highest number of earthquakes

7) Find the top 10 states / countries where the strongest and weakest earthquakes occured

It looks like US states are being treated differently from foreign countries. We would like to fix that.

How can we tell if a name is a US state name? Python has a package for that:!

This is a good time to try installing a new package using pip. Pip is the original python package manager that predates conda. Basically conda is more oriented towards data science while pip is more general purpose. There are lots more packages on pip than on conda. You can read a comparision of these two utilities if you want to know more.

8) Install the us package using pip, either directly from the notebook or the command line

The shell command is pip install us.

9) Import the us package to verify your installation works

10) Read the us documentation to figure out how to create a list of state names (all upper case)

11) Write a function to check whether a string is a US state name.

This function should not be case sensitive. It should also strip any whitespace out of the test string.

12) Use Pandas' apply function to apply this to each row of the country name series

You should produce a new boolean series. (Hint: use dropna() to drop rows that have null values before calling apply.)

13) reindex this boolean series to match the dataframe's index

Fill the null values with False using .fillna().

14) Now re-assign the country column in the DataFrame to USA if the row is a state.

Also add the state name as a new column.

15) Now redo the country count and minimum magnitdue using the corrected data

16) Create a filtered dataset that only has earthquakes of magnitude 4 or larger

17) Analyze the distribution of the Earthquake magnitudes in the filtered distribution

Make a histogram of the Earthquake count versus magnitude. Make sure to use a Logarithmic scale. What sort of relationship do you see?

fig, ax = plt.subplots() df_filt.hist('mag', bins=20, ax=ax) ax.set_yscale('log')

18) Visualize the locations of earthquakes by making a scatterplot of their latitude and longitude.

Use the filtered data. Color it by magnitude. Make it pretty

19) Do the same for depth