Thursday, September 06, 2007

Data mining = Evil?

Some get a chill when they hear "data mining" because they associate it with "big brother". Well, here's one more major incident that sheds darkness on smart algorithms: The Department of Homeland Security declared the end of a data mining program called ADVISE (Analysis, Dissemination, Visualization, Insight and Semantic Enhancement). Why? Because it turns out that they were testing it for two years on live data on real people "without meeting privacy requirements" (Yahoo! News: DHS ends criticized data-mining program).

There is nothing wrong or evil about data mining. It's like any other tool: you can use it or abuse it. Issues of privacy and confidentiality in data usage have always been there and will continue to be a major concern as more and more of our private data gets stored in commercial, government, and other databases.

Many students in my data mining class use data from their workplace for their term project. The projects almost always turn out to be insightful and useful beyond the class exercise. But we do always make sure to obtain permission, de-identify, and protect and restrict access to the data as needed. Good practice is the key to keeping "data mining" a positive term!

No comments: