An adaptive agent for Google Place crawling

EasyChair Preprint 934, version 1

Abstract

This agent was developed over the question "how to crawl all google place of an urban area?". To solve this i developed a data crawler for Google Place as and adaptive agent. This was used to crawling data in the Master Thesis "Geospatial data analysis for Urban informatics applications: the case of the Google Place of the City of Milan". In this context the urban area is a real envrioment while the Google Place API is a digital representaion of this, both have some limitations and rules to access it. In this case our envrioment is Google Place API while the agent goal is capturing all Places of an area with a minimum input. This agent in completely autonomy, whit a minimum input, capture all Place by center position to a maximum diameter, both specified by user. This is realized over a spiral movement inspired on my Roomba spiral-pattern. In this case the cells where the agent moves are hexagons, why this are best approximation of circle and has the property to have same distance by neighboring hexagons. The agent work on three steps: Planning of track with cells of crawling; Process of crawling over cells; Checking of results and if necessary replanning of track of crawling in more fine cells. The minimun user-input are composed by: center of crawling, default size of cells and finally the number of spirals of crawling. The algorithm choose when use more smaller cells and where, and, if there are some problem, where re-planning cells. So the core of algorithm is the adaptation on some enviromental details of his planned track and granularity tesselation previously planned.

Keyphrases: adaptive agent, geospatial, social media, urban analysis

@booklet{EasyChair:934,
  author    = {Domenico Monaco},
  title     = {An adaptive agent for Google Place crawling},
  howpublished = {EasyChair Preprint 934},
  year      = {EasyChair, 2019}}