Scrambling real-time location data of the population over time
This is not a typical question asked on StackOverflow, but I couldn't think of a better place, so I apologize in advance.
I'm looking for some kind of API or other solution that allows me to constantly clean up the population location (~ 50 + users ideally) in semi-real time. To clarify, I would like the GPS sampling to be fairly regular (~ hourly max), but I'm good at getting data late. I know services like Snapchat and Facebook have data like this and expose it to users at times, but I can't find any good way to clean it up. The data does not need to be tagged with any user ID.
I understand that I could create some kind of mobile app to collect this data for me, but I was hoping there would be a smarter solution that would allow me to clear this information without having to go through hoops. Thank.
source to share
After extensive research and personal testing, your query is a little more complex than you might expect. I will try to explore all the possibilities that are relevant to you, and others may adjust or criticize my answer as needed.
There is actually no service to clean up anonymous location data at your request, and a big reason for this is a combination of laws and privacy. There are several sources here, not necessarily authoritative, but the following problems are posed:
https://news.mit.edu/2013/how-hard-it-de-anonymize-cellphone-data
https://myshadow.org/location-tracking
https://www.law.berkeley.edu/wp-content/uploads/2015/04/2016-06-07_Cell-Tracking-Primer_Final.pdf
The articles discuss how easy it is to connect anonymous location data for individuals, and the links they link to MIT are about research done on location databases: 95% of the 1.5 million people were linked to tracking data cell phone for identification. This is kind of scary, and if such information were publicly available, the confidentiality outrage would be enormous.
Regarding the Berkeley laws pdf, individual data is of course only available to law enforcement. These issues have led to privacy policies of companies like Google and Apple, where location information for an app is strictly regulated by that app and the manufacturing company. Public publication of such information is prohibited. Check out their privacy policy for more information.
It boils down to this: if you want to track the locations of people, even if it's a small pool, you need to create a facility to which you are granted special permission, the application they use, and the company and location of the API that the application uses. Snapchat and Facebook can do this and share this information because they have a privacy policy and that the people using their software agree and the infrastructure to receive this information in real time. If privacy is abused, it is now the responsibility of the users, not the company providing the service. And all this information is strictly limited by the users of this application and in the context that they mean, that is, only friends can see the information.
From this long premise, you really only have two options.
- Create an app that users can download, or you can share this app with people and they can share their location to send you "anonymous" location data. Then add that to the database or use it in some other way to get the information you are looking for.
- Be a researcher at a major technical school (like MIT) or other research organization and ask for or participate in a research project that includes location data.
The point is that location is sensitive and this information is not publicly available. You must provide funds or go through the company that provides this information. I would add API examples for building an application, but there are so many different devices, programming languages, and methods for doing this that could fit better in his own question.
Again, if there is information that I missed, feel free to add it.
source to share
You can see the API here.
https://www.programmableweb.com/news/51-gps-apis-breadcrumbs-ipoki-and-waze/2013/10/16
Also, some sample data can be found here
http://tunedit.org/challenge/IEEE-ICDM-2010?m=summary
http://archive.ics.uci.edu/ml/datasets.html
https://archive.ics.uci.edu/ml/datasets/GPS+Trajectories
Sounds like an interesting project ... good luck.
Additional questions for you: You mentioned scraping, can you specify the location of the data you want to scrape so that a scraping solution can be provided?
As for the smart solution: You can create a simple web page with the current time at the users' location. While the user is opening this page, you can use a realtime javascript solution like the one described here https://www.pubnub.com/tutorials/javascript/mapping-javascript-tracking/ or just add it to an existing website. Create ajax connection to send user location to server. Of course, this requires a long time for the user to open your web page.
Alternatively, if you have shared your purpose, then perhaps the invited audience will join in to help provide this information for you and activate the application on their mobile.
source to share