9/19/2011

Software that predicts the future: Does it really help?

Shelley Metzenbaum, associate director of performance and personnel management at OMB, speaks at a conference in Washington, D.C. (Army Times Publishing Co.)


Πηγή: Federal Times
By SEAN REILLY
Sep. 18 2011


For the IRS, a type of software known as predictive analytics may offer one means of deciding which taxpayers most likely warrant an audit. The U.S. Postal Service's inspector general has just begun using analytics to identify high-risk contracts. And at least one agency is exploring the technology as a means of predicting which of its employees will be retiring soon.

The approach, heavy on mathematical modeling and long employed by business, is drawing increased attention from government, both to boost efficiency and as a way of making better use of its vast stocks of data to help predict the future.

"Potentially, it's dynamite," Richard Huang, an information technology specialist at the Veterans Affairs Department said last week.

But the return on investment is not easy to measure. Even some vendors worry that the software's potential may be overhyped in some cases.

"It still requires common sense and hard work," said John Elder, chief scientist at Elder Research, a data-mining consultant that helped to organize a conference last week on the government's use of predictive analytics software.

Predictive analytics can be useful to an agency "savvy enough to know how to use them and apply them," said Douglas Samuelson, president of InfoLogix, a Virginia consulting firm. "Unfortunately, in some agencies, you don't get to making better decisions; you're too busy trying to get to any decision."

Predictive analytics in essence means crunching data from past events to predict future behavior. The technique is widely employed by credit card companies to target fraud; supermarket shoppers encounter it when a cash register spits out a coupon that — based on warehoused information about individual purchasing habits — pitches discounts for products they're likely to buy.

While no one tracks federal spending on analytics, contractors report a surge in interest.

At North Carolina-based SAS, for example, government customers make up its second biggest software revenue stream, although still well behind clients in the financial service industry.

One fan is the White House Office of Management and Budget, which is encouraging agencies to use analytics to improve results for taxpayers, communities and businesses. Shelley Metzenbaum, associate director for performance and personnel management, said OMB is working to identify and share effective analytics practices.

Other factors are also driving interest in this emerging technology. For one thing, agencies have been gradually automating more of their administrative operations in the last decade, meaning they are accumulating large volumes of electronic data that they can now subject to analysis. In addition, ever-tightening budgets are pressing agencies to do better at prioritizing where they direct their time and resources — something at which predictive analytics is especially helpful.

Also, one high-profile federal program — the Recovery Accountability and Transparency (RAT) Board, created two years ago to monitor hundreds of billions of dollars in grants, loans and other spending under the stimulus bill — has applied predictive analytics with much success.

Instead of resorting to traditional "pay and chase" methods of pursuing fraud after the fact, the RAT Board fashioned a new model that mines information from an array of databases to ferret out problem contractors and head off bad behavior from the start.

So far, the recovery program has been largely scandal-free, prompting a bipartisan push for its expansion to other areas of federal spending, something President Obama has started exploring.
Seeing success

At the IRS, which processed almost 231 million income tax returns last year, "technology is changing how we can do business," Patricia McGuire, the agency's deputy director of research analysis and statistics, said at last week's conference.

In a pilot project started last year, the IRS is using analytics to better focus its audits on taxpayers most likely to owe money than those already obeying the law. The process of determining where to aim the agency's audit teams is complex, involving risk scores, algorithms and statistical modeling. So far, however, the project has registered drops in "non-productive" audits of up to 50 percent, said John Kam, a senior analyst at IRS.

The Securities and Exchange Commission, which is picking up a bundle of new regulatory responsibilities under the Dodd-Frank financial overhaul approved last year, has a similar need to choose wisely where it directs its resources.

The agency has roughly 600 examiners to oversee some 11,000 private-sector investment advisers, said Harvey Westbrook, assistant director of SEC's Office of Quantitative Research in the Division of Risk, Strategy, and Financial Innovation.

"Analytics really help allocate your manpower resources much more effectively," Westbrook said. Some big investments will be needed to keep building on those initiatives, he said.

The cost for these software products is a challenge, said Ed Slevin, director of the Computer-Assisted Assessment Techniques Division at the Education Department's Office of Inspector General. Another is assembling and compiling the data.

For one federal project aimed at predicting retention in an agency's workforce, more than 7 million rows of data covering three years of payroll and other records on more than 100,000 employees were needed, said Dave Vennergrund, director of business analytics at Delta Solutions and Technologies. He declined to name the agency at his customer's direction.

Also key is ensuring that the final product is tailored to the needs of users. At the Postal Service IG's office, which recently rolled out a new system that assigns risk-based scores to thousands of contracts, investigators can access the information simply by clicking on a map of the United States.

"Understand the power of visualization," said Bryan Jones, director of the data mining group in the IG's Office of Investigations. "If we can't translate it easily to our users, we've wasted a lot of time and a lot of effort and a lot of money."


No comments:

Post a Comment