It’s been six months since I launched the Oakland Crime API. Since then, my scrapper has been pulling data from CrimeWatch and storing it into a database. I started this project because CrimeWatch keeps only the last three months of crime reports on any given day. This makes year-over-year crime analysis difficult. To be fair, you can get the last year’s crime reports in a single CSV file, but that is released once a year.

(By the way, why no updates in the past six months? A week after I launched, I started a new job at Fitbit as a Senior Product Analyst. I wanted to focus on ramping up before getting back to this site.)

The data

Presently, there are over nine months of organized data in the database. Remember, CrimeWatch stores only the last 90 days. Some charts:

Daily cases

There’s a seasonal cadence to crime reports, which you’ll see is mostly due to property crimes rather than violent or drug.

The big drop after July is why I’m doing this. I discussed before how there’s a several week data lag from CrimeWatch. To me, this lag breaks a major use case of any crime mapping website. If recent crime is half-reported, what’s the use of visualizing it? This is why I’m storing the data in a warehouse.

Daily cases by type

Breaking the crimes by type, a few things emerge:

  1. The vast majority of case reports are property-crimes, to the tune of 64% in the past nine months. A quarter of the reports were violent crimes. Drugs accounted for 3%
  2. No one type seems to be rising. I wonder if I normalized the data by population, the crimerates by type would be dropping
  3. The Eric Garner protests had the largest single day crime reports in the last nine months

What’s next?

In about three months we’ll have completed a full year’s worth of data, which will let us talk about annual crime rates. By then, I want to put this data into a daily-updated dashboard that anyone can view. I want to make a dashboard that will tell you, for instance, what is the current rate of property crime, and how it compared to a week/month/year ago.

I hope as more data is collected, we’ll be able to say with confidence which way Oakland crime is trending.


comments powered by Disqus