More than 60% of eCommerce organisations believe that real-time data collection will be their primary focus in 2023, according to a new report from Oxylabs.
In conjunction with Censuswide, Oxylabs surveyed more than 1000 senior data decision-makers from eCommerce companies, gathering insights on attitudes towards public web data collection in the sector.
When asked about their goals for collecting data in the coming year, 62% of eCommerce organisations said that real-time data would be at the forefront.
According the report, as most web pages today are subject to frequent content changes, real-time web scraping is an essential function for any digital business.
In addition, real-time web scraping will be increasingly important to eCommerce businesses in the future, allowing them to track information on their competitors' websites and other online stores, it says.
As companies begin to rely more on efficient and low-cost data collection methods such as external data gathering, used by 40.54% of respondents, there is a clear shift towards gaining more actionable insights.
“These goals might indicate that businesses have yet to harness the true potential of real-time data. Greater data delivery speeds and instant insights can prove itself to be the next edge of competition that eCommerce companies are faced with," says Tomas Montvilas, Chief Commercial Officer at Oxylabs.
"As competitors adopt real-time data for purposes such as monitoring, others will want to follow suit, causing a feedback loop. I expect to see real-time data keep climbing in importance in the coming years," he says.
Montvilas says these trends show that eCommerce companies must consider new challenges in their business environment. For applications with low analytical requirements, such as dynamic pricing, real-time data is most valuable.
Meanwhile the most widely used use case for external data was cited as market research—which necessitates high levels of involvement with the data.
According to the report, US-based companies will focus more on collecting and utilising external data (43.91% vs 37.20%), whereas their non-US counterparts will pay closer attention to managing existing infrastructure—56.09% vs 44.49%. On the other hand, UK-based companies are planning to focus more on real-time acquisition (65.94% vs 58.28%) and data quality assurance (28.35% vs 18.96%). This contrast shows that the two regions have unique web scraping environments, as each one faces specific issues in this regard.
“As eCommerce has grown exponentially, businesses are increasingly dependent on data," says Montvilas.
"Today's organisations feel compelled to measure everything from the number of transactions per hour to the lifetime value of their customers," he says.
"Web scraping has become an essential part of all decision-making processes for businesses large and small, which means that eCommerce companies can now use real time data to gain a competitive edge in difficult economic times."