Data Mining Crime Correlations Using San Francisco Crime Open Data

Transcription

1 Data Mining Crime Correlations Using San Francisco Crime Open Data Kiel Gordon Matt Pymm John Tuazon California State University Sacramento CSC 177 Data Warehousing and Data Mining Dr. Lu May 16, 2016

2 Abstract: San Francisco has open data sets online that are published by the City and County of San Francisco for the purpose of transparency and data analysis. Using crime data from SF OpenData we will explore data mining techniques with the purpose of using classifiers to understand the resolution of a crime or where a crime occurred. We used tableau to visualize data, and Weka and the naive bayes algorithm to classify the data. By using classification of crime resolutions and knowing where the offenses occurred, we can see how effective individual police districts are at booking and resolving crimes. Overview and Background: Data mining is the process of extracting knowledge and patterns from a large amount of data. It is the analysis part of the Knowledge discovery in databases also known as KDD. The knowledge we attempted to extract from our 1.9 million row data set pertained to patterns of crime throughout the San Francisco Police Districts. We found our data on SF OpenData. It contains records of all crimes reported to the SFPD from It was a 1.9 million rows CSV with 13 dimensions. Dimensions include GPS, type of crime, the resolution, address of the crime, date, time, police district, and day of week. The data are Incidents derived from SFPD Crime Incident Reporting system and they are updated daily, showing data from 1/1/2003 up until two weeks ago from current date. Motivation Our connection and interest in San Francisco is from it being one of the first cities to have an open data policy. This has lead to numerous other cities to adopt similar policies and massive amounts of data to now be available. In addition, San Francisco is a popular place for tech jobs, one of our team members is considering jobs in the San Francisco area. This incentivized the team to utilize San Francisco open data to aid people who plan to move into San Francisco with crime information that can help them in their relocation decision. Current and future residents of San Francisco could utilize our results to determine what category of crime most often occurs in their district, and how well their district police force resolves crime. Current and future San Francisco residents having prior knowledge of this information could better prepare themselves for preventing these incidents from occurring to them or even allow them to live safer lives by providing them knowledge of the surrounding crime. Many people who move to San Francisco are new to cities and are vulnerable to crime from being unacclimated to their surroundings. If a region a resident regular runs in tends to be high in kidnappings, this information would allow them to reconsider their choice of running location, and provide them safer alternatives. 1

3 The San Francisco open dataset allows for exporting data in many formats, with the choices from CSV, to JSON, to XML. The socrata platform that the data is housed has the ability shows all 1,907,487 rows. For our data mining we chose to use one data set. Originally we wanted to compare Sacramento s crime data to San Francisco crime data but the Sacramento Crime data set differed from San Francisco s as they classified crime differently and that was the main criteria that we wanted to mine the data over. San Francisco s open data was clean and well maintained, and had included all of the information that our data mining required. Objective Our data mining objective was to understand crime in San Francisco and predict answers to questions that were inspired by our data mart. Our previous work on the data mart allowed us to understand the common crime patterns in San Francisco as well as granting us guidance into expectations of the data mining results. By using the other 12 attributes of the data set as predictors which included IncidentNum, Category, Description, DayOfWeek, Date, Time, PdDistrict, Resolution, Address, X coordinate, Y coordinate, GPS Location, and PdId we generated the three questions we intended to answer. The three questions were 1. Can we predict the category of crime? 2. Can we predict the resolution of a crime? and 3. Can we predict which Police Department District the crime will occur in? By answering these three questions we can provide future and current residents of San Francisco with crime information which can aid them in making their decision between where to move and knowledge of their surroundings. Data Mining Design and Methodology Design We used our Data Warehouse results to drive our Data Mining questions we wanted to answer. The three questions we intended to answer above were addressed using Weka and RapidMiner. We used Tableau to visualize the information for easy interpretation for the audience and reader. Our data mining process consisted of two primary approaches. The first approach was done with Weka on a representative subset of 20,000 examples while the second approach performed in RapidMiner utilized the entire population of the data. On our first approach we confronted several limitation with Athena in both processing and permission issues, when using Weka on CSUS ECS computers. In response to this, we decided to use a representative subset of 20,000 examples from our original 1.9 million examples. In our second approach using RapidMiner, we were able to utilize a desktop computer that had 24GB of RAM which allowed us to analyze all 1.9 million examples. Methodology For both approaches Weka and RapidMiner we used Naive Bayesian classification algorithm to predict answers to our three questions. Naive Bayesian was favored by the team by it calculating the probability by counting the frequency of values and combinations of values in historical data, it seemed a perfect fit for our application. In addition, another reason for the team choosing 2

4 Naive Bayesian Classification stems from the massive data set that we had. Naive Bayes makes assumptions all examples are conditionally independent and the predictors are independent as well. If Naive Bayesian assumption is a mistake, it has the benefit of not significantly impacting the accuracy or kappa negatively. Naive Bayesian allowed us to reach high accuracy and allowed our data set to be computationally feasible. If we used a non assumption based algorithm it s likely the data set would of been computationally infeasible which would of prevented us from performing any analysis. When using Naive Bayesian in RapidMiner and all 1.9 million examples of data we were able to obtain high accuracy and kappa as mentioned above. Therefore we experimented with running 10 fold cross validation as well as 20 fold cross validation to measure the difference in accuracy and kappa. The results from both tests were the same. This allowed us to form a conclusion that 10 fold cross validation is the best for computation power and accuracy and is why RapidMiner defaults any cross validation to 10 folds. The team utilized 10 fold cross validation throughout the rest of the data mining analysis. Results Predicting Crime Category In RapidMiner We attempted to predict category given the other 12 attributes using all 1.9 million examples in RapidMiner. We were interested in crime ranking and if we could predict most committed and least committed crime accurately from our data set. We expected the accuracy to be low for category because there is 39 different options. By merely guessing any of the 39 categories that is (1/39) * 100 = 2.56% accuracy. Our results were surprisingly accurate at 95% accuracy and kappa. We were interested in which attributes contributed the most to the prediction. So we built a decision tree in RapidMiner on our 1.9 million data set to determine the influences each attribute had on the prediction. Resolution and date provided the most influence on the determining category. This indicates categories generally correspond to certain resolutions, and are affected by the date. Figure 1 (Left): Category Confusion Matrix Figure 1 illustrates our predicted results using patterns found from the other 12 attributes to predict the category of crime 3

5 for each of the 1.9 million examples. These result are almost exactly in line with what we predicted and inferred from our data warehouse. As illustrated in Figure 1 in the confusion matrix using RapidMiner s PlotView, the majority of the high frequency predictions occur along the diagonal, indicating a true positive and correct classification. The diagonal is labeled from 1 to 39 indicating each crime category. On the list on the right of Figure 1, six categories are listed showing their number in the diagonal. On the left of the Figure 1 the top four categories are shown. Larceny and theft has the highest occurrence, followed by other offenses. Other offenses includes any offense not included in the other 38 categories. The least committed crime was treason. Our predictor classified all treason incorrectly. This trend occurred throughout our data mining experience, and we concluded if provided too few examples, the classification algorithm does not have enough knowledge to know what treason is. Value of This Result On Current and Future Residents of San Francisco The top three categories of crimes were nonviolent which should be a relief for San Francisco residents. San Francisco residents should be aware of property because the top crime category was theft of personal property with 387 thousand incidents. RapidMiner showed some statistics after importing the data, and the most common crime was at 12 am, and the least common was 5:32 am, so night is the prime time for crime. Other offenses and non criminal offenses were second and third which is actually a good sign, because both of these are minor offenses and nothing severe. Other offenses consists of traffic violations, license expirations, or other minor incidents. From our analysis spanning 14 years from 2003 to 2016, and having a high kappa, we can confidently predict the top three crime categories will remain the same for years to come, which should come as some relief as they're much worse crimes that could top the list. If given more time, the team was interested in pursuing crime category classification with less prediction attributes in an attempt to be able to predict crime categories from only the time, date, X and Y coordinates, and GPS location. This could enable police to narrow in on temporal crime zones and dynamically allocate police presence to crime hot spots throughout the day. This knowledge from patterns in those attributes would reduce the amount of crime by allowing police presence at the right spot at the right time potentially saving residents lives and stopping crime when the incident occurred instead of responding to it after the fact. We will get into police districts which are hotspots for crime and the resolution to these incidents in the following sections. Predicting Crime in Police Department Districts in RapidMiner After extracting knowledge and answering our first question What is the most and least committed crime?, we attempted to predict What Police Department District did the crime occur in? given the other 12 attributes and all 1.9 million examples in RapidMiner. After training and validation we obtained high accuracy and kappa 99.2% and.992 respectively. We utilized RapidMiner s Confusion Matrix PlotView to analyze our results. 4

6 Figure 2 (Above): Police Department District Confusion Matrix Figure 2 illustrates our predicted results using patterns found from the other 12 attributes to predict the Police Department District an incident occurred in for each of the 1.9 million incidents. These result are almost exactly in line with what we predicted and inferred from our data warehouse. As illustrated in Figure 2 in the confusion matrix using RapidMiner s PlotView, the majority of the high frequency predictions occur along the diagonal, indicating a true positive and correct classification. The diagonal is labeled from 1 to 10 indicating each Police Department District in San Francisco. On the list on the right of Figure 2, ten Police Department Districts are listed showing their number in the diagonal. On the left of the Figure 2 the top three crime ridden Police Dept. Districts are shown as well as the bottom three. San Francisco Southern district had the most crime at 339,702 incidents from 2003 to The next two districts Mission and Northern were close in number of incidents with 256,154 and 227,518 respectively. The least crime districts Richmond and Park had less than ⅓ of the crime that occurred in Southern and half the crime of Mission and Northern. Value of This Result On Current and Future Residents of San Francisco Current and future residents should be aware much higher crime occurs in the Southern, Mission and Northern districts. If residents live, work or travel through these districts they should be aware of their surroundings. Current residents may consider moving to the least crime Police Department Districts such as Richmond, Park or Taraval. Richmond and Park have ⅓ the crime Southern has, and ½ the crime Mission and Northern have. By our prediction being based off crime data from 2003 to 2016, a 14 year span, crime in these top three districts have a high probability of continuing to top the charts as well as the lowest three districts remaining much less crime ridden in the years to come. If given more time, the team was interested in pursuing Police Department District classification with less prediction attributes in an attempt to be able to predict specific categories of crime in Police Department Districts at specific times of the year. These predictions would of only utilized six attributes category, time, date, X and Y coordinates, and GPS location. This would 5

7 contribute to generating the temporal crime zones as discussed in the category section and allowing dynamic allocation of police presence to crime hotspots throughout the day based off historical trends in crime patterns. We will get into resolutions to these incidents in the next section. Predicting Resolution of Crime in RapidMiner After providing answers to the first two questions, we looked to provide an answer to our third question How was a crime resolved?. As in the previous predictions, we used the other 12 attributes and all 1.9 million examples in RapidMiner. We were interested in the resolution ranking and if we could predict the most resolved and least resolved crime accurately from our data set. We achieved moderate accuracy and kappa from our predictions, 70% and.53 respectively. Figure 3 (Above): Resolution Confusion Matrix Figure 3 illustrates our predicted results using patterns found from the other 12 attributes to predict the Resolution of an incident for each of the 1.9 million incidents. We anticipated the ranking of the resolution from understanding the crime patterns during the creation of our data warehouse. As illustrated in Figure 3 in the confusion matrix only a few high frequency predictions occur along the diagonal, unlike the previous confusion matrices shown in Figure 1 and Figure 2. The diagonal is labeled from 1 to 17 indicating each of possible resolutions that may occur from the incident. The top three resolutions are no resolution to an incident, arrested / booked, and arrested / cited. The number in parentheses following the ranking is our predicted number, the following number in parentheses is the actual number. The highest resolution of 6

8 crimes in San Francisco is none with 1,160,691 incidents going unresolved. Two times more incidents go unresolved than the second resolution arrested/booked, and more seven times the incidents go unresolved than the third resolution arrested / cited. We concluded the reason the accuracy and kappa for resolution was much lower than category and Police Department District was many of the resolutions are court related. When the resolution was directly crime related such as none, arrested / booked, arrested / cited the prediction was fairly accurate at around 70%, however with other resolutions such as district attorney refused to prosecute, prosecuted by outside agency, predictions were very low because those type of resolutions are impossible to predict from an incident with the provided attributes and are also not directly related to the crime. Value of This Result On Current and Future Residents of San Francisco Our analysis and predicted answer to How are crimes resolved? provides residents of San Francisco with knowledge on the most common resolutions of crime. This knowledge may encourage residents to contribute to the effort as a community and help contribute to the resolution of crimes through providing any knowledge or information they may have regarding an incident. The bad news is the top resolution is none, and this generally is true for each crime category. If none is not the top resolution for a specific crime, it is generally in the top three resolutions for that crime category. This resolution shows the San Francisco police force is confronted with too many crimes, overwhelmed and need additional assistance. Our predictions are based on years 2003 to 2016, over a 14 year span, therefore it is highly unlikely there will be a shift in the top resolution statistics in a much more positive direction in the years to come unless technology such as advanced analytics or the community contribute to the crime effort give the advantage back to the law enforcement. Summary Crime in San Francisco is out of control. Police need assistance either through community contribution or technological assistance such as advanced analytics to help predict and assist with crime, to enable police to narrow in on temporal crime zones and dynamically allocate police presence to crime hot spots throughout the day. This knowledge from patterns in those attributes would reduce the amount of crime by allowing police presence at the right spot at the right time potentially saving residents lives and stopping crime when the incident occurred instead of responding to it after the fact. Majority of crimes are theft, traffic incidents, and civil incidents however each crime category has a resolution of none as the top resolution or at least in one of the top three resolutions. Current or future San Francisco residents who want to avoid crime clusters should move to lesser crime ridden districts such as Richmond, Park, or Taraval. Most crime occurs at night between the hours of 12 am to 5:32 am. Our prediction results overall performed well with high accuracy and high kappa showing strong stability, which indicates the predictions made here should hold for several years to come. 7

9 Learning Experiences We had a difficult time narrowing down on which questions to focus on predicting with our data mining, by there being so much data and so many potential questions. Therefore we used our data mart to guide our questions we intended to answer with our data mining. Running Weka from the school computers on the athena server forced us to make a representative subset of 20,000 so it could be processed. Running all 1.9 million examples through RapidMiner provided very close results as the representative subset of 20,000 examples showing accuracy does not merely depend on the volume of data but also the quality of data. Naive Bayesian is a powerful classification algorithm that allowed us to obtain high accuracy and kappa as well as allowing us to process all 1.9 million examples. Through our questions we answered and our data mart, we have gained a deep understand of crime trends and patterns that occur in the City of San Francisco and how future trends will occur. Main Results Recap In predicting the answer to the question, What is the most and least committed crime? with RapidMiner from 39 categories of crimes, our results were surprisingly accurate at 95% accuracy and kappa. We provided value to San Francisco current and future residents by revealing the top three category of crimes were nonviolent. This should be a relief to San Francisco residents. We discovered the top crime category was theft of personal property which had 387 thousand incidents. The second and third were other offenses and non criminal respectively. From our analysis spanning 14 years from 2003 to 2016, and having a high kappa, we can confidently predict the top three crime categories will remain the same for years to come, which should come as some relief as they're much worse crimes that could top the list. Next we predicted What Police Department District did the crime occur in?, again with high accuracy and kappa of 99.2% and.992 respectively. We provided value to San Francisco current and future residents by providing the top three crime districts Southern, Mission and Nothern, along with the lowest three crime districts Richmond, Park and Taraval as safer alternatives.by our prediction being based off crime data from 2003 to 2016, a 14 year span, crime in these top three districts have a high probability of continuing to top the charts as well as the lowest three districts remaining much less crime ridden in the years to come. Finally, answering the third and final question How was a crime resolved? with a moderate accuracy and kappa of 70%, and 0.52 respectively we predicted the resolutions of crimes over the 14 year span. We provided value to San Francisco residents with knowledge of the most common resolutions of crime. The bad news is the most common resolution by a significant amount was none, which generally resides in the top three resolutions of most crimes. This shows San Francisco police force is overwhelmed and needs more assistance from either more police or community effort. Our predictions are based on years 2003 to 2016, over a 14 year span, therefore it is highly unlikely there will be a shift in the top resolution statistics in a much more positive direction in the years to come unless technology such as advanced analytics or the community contribute to the crime effort give the advantage back to the law enforcement. 8

10 References [1] Textbook: Jiawei Han, Micheline Kambe, "Data Mining", 2nd Edition, [2] Prof. Lu, CSc 177 Lecture Notes, Spring Website: [3] SF OpenData, SFPD Incidents from 1 January Website: Safety/SFPD Incidents from 1 January 2003/tmnf yvry Appendix Below is an example of the visualizations we created with Tableau. You can view crime trends by the Category of crime as well as by Police Department District. It will also show you at which location in San Francisco a crime took place. We created one of these for every year between 2003 and

11 Useful Resources Used During The Project Recommend Online Resources Newly Released RapidMiner Tutorials Great Tutorial On Most Programming Languages: Used for PHP, MySQL, and Javascript Great Data Science Community Bootstrap Framework: Great for making mobile responsive websites with professionally written CSS and Javascript libraries. Javascript Charts: We almost used this resource and wanted to share it. Helpful Web Development Tutorials that were used during the Data Warehouse Project (We used it for PHP, and Javascript) 10