Data Mining in the Telecom industry

A concise paper on how data mining is used in the telecom industry to detect fraud, customer churn, marketing opportunities and also predict network failures. Telcos gather enormous volumes of data in the form of Call Detail Records (CDR's), customer information and also network messages (from all the switches and other equipment). The CDR and network status data are too detailed to be of direct use in data mining and hence the first step is to summarize this data along some useful features and to then feed this to a data mining algorithm or tool like SAS. The challenge here is that the events that one is trying to predict are all very rare events (with less than 0.5% probability) and this has to be extracted from billions of rows of data - truly finding a needle in a haystack! Such stuff is just not possible to do manually or programmatically or by specifying some analytical rules - the data has to be mined to generate both the rules and their probability (or confidence). Interesting area...