Humans account for less than 40% of Web traffic

Who's actually surfing the Internet? Lots and lots of bots, according to a new report, including benign search engine crawlers as well as malicious 'scrapers' and spammers.

By MSN Money Partner Dec 16, 2013 2:19PM

Venessa Salazar, right, helps her mother, Lisa Salazar, use a computer at a public library to navigate the Affordable Health Care Act website in San Antonio. © AP Photo/Eric GayThe Fiscal Times on MSN MoneyBy Yuval Rosenberg,
The Fiscal Times

The Internet is a sprawling and mysterious place, as anyone who's spent two minutes on Google (GOOG) -- and certainly anyone involved with web publishing -- can attest. It's a thrumming hive filled with vast amounts of data, knowledge, commerce and media and an equally vast number of gloriously bizarre arguments, grumpy cats and adorable sloths.

Also, bots. Lots and lots of bots.

Bots are software applications built to perform automated tasks. They can be valuable, as with search engines crawling and indexing websites. They can also be malicious, like those used by hackers and spammers.

A report released this week by Incapsula (IMPV), a cloud-based web-security service, found that 61.5 percent of all website traffic now comes from these non-human visitors. If you're reading this and you're human, you're apparently in the minority.

Incapsula based its findings on 1.45 billion bot visits, over a 90-day period, to 20,000 sites on its own network. That network may not be perfectly representative of the entire web, but it offers a sense of who, or what, is out there. (View an infographic of the results here.)

Bot traffic is up by a fifth from last year, according to Incapsula. Most of that growth comes from "good bots," like those from analytics companies and search engines that are crawling sites more frequently. Legitimate bot activity has jumped 55 percent, which Incapsula suggests is largely the result of new online services and increased crawling by search engines seeking the timeliest results. Those search engines and other good bots account for 31 percent of web traffic.

Another 30.5 percent of traffic comes from malicious bots, including "scrapers," which look to steal content or email addresses; "hacking tools"; a shrinking pool of spammers; and a growing group of what Incapsula calls "other impersonators" and describes as "unclassified bots with hostile intentions.

"Malicious bot traffic as a percentage of overall traffic hasn't changed, but the specific kinds of bot activity has shifted, Incapsula said. Spam bots have decreased from two percent of traffic in 2012 to 0.5 percent this year. "The most plausible explanation for this steep decrease is Google's anti-spam campaign," the report said.

Activity by more sophisticated impersonator bots -- "the tools of top-tier hackers who are proficient enough to create their own malware" -- has risen, though. These impersonators are usually custom-made for a specific activity that involves penetrating a website's security, according to the report. "The 8 percent increase in the number of such bots highlights the increased activity of such hackers," Incapsula said, "as well as the rise in targeted cyber-attacks."

More from The Fiscal Times

Dec 16, 2013 4:55PM
What percentage of those were just searching for robot porn?
Dec 16, 2013 2:32PM
Soon, the terminators will appear! (looking like BHO)
Dec 16, 2013 6:34PM
Makes sense. Does anyone really think someone at Amazon is banging away on a keyboard, thanking them for their order?
Dec 16, 2013 4:47PM
The internet has always been mysterious to me. 
Dec 18, 2013 1:37PM
I for one welcome our new robot overlords.
Please help us to maintain a healthy and vibrant community by reporting any illegal or inappropriate behavior. If you believe a message violates theCode of Conductplease use this form to notify the moderators. They will investigate your report and take appropriate action. If necessary, they report all illegal activity to the proper authorities.
100 character limit
Are you sure you want to delete this comment?


Copyright © 2014 Microsoft. All rights reserved.

Fundamental company data and historical chart data provided by Morningstar Inc. Real-time index quotes and delayed quotes supplied by Morningstar Inc. Quotes delayed by up to 15 minutes, except where indicated otherwise. Fund summary, fund performance and dividend data provided by Morningstar Inc. Analyst recommendations provided by Zacks Investment Research. StockScouter data provided by Verus Analytics. IPO data provided by Hoover's Inc. Index membership data provided by Morningstar Inc.


Start investing in technology companies with help from financial writers and experts who know the industry best. Learn what to look for in a technology company to make the right investment decisions.





Quotes delayed at least 15 min