Jaspersoft’s new big data survey includes 631 respondents from the company’s user community. The survey includes respondents from more than fifteen countries that are primarily employed by companies with less than US$ 10M in revenue (30 percent).
In addition, most of the participants indicated they were in technical roles. Only 6 percent of respondents specified they were business users, while 63 percent were application developers and 19 percent were either report developers or business intelligence administrators. The high number of respondents in non-management roles is important to note because there is a risk it could skew the results. The participants may have detailed knowledge of implementation details, but may lack visibility across the enterprise to all big data initiatives that are underway.
Even if participants don’t have insight into what’s swirling in executives heads, they are aware the work on managing big data has started. According to the survey, twelve percent of the companies represented have already deployed a big data analytics solution. Twice as many, 24 percent, are currently implementing, and 13 percent plan to have a project underway in the next six months. Another 13 percent are planning a project in the next 12 months. However, a significant number of organizations, 38 percent, have no immediate plans for initiating a big data project.
Given all of the data that suggests big data can yield enormous business benefits, why are 38 percent of the companies represented choosing not to invest in big data? Respondents cited multiple reasons, but the most prevalent response (37 percent) was that their organization only had structured relational data. The second most popular answer, 35 percent of responses, was that the organization did not understand what big data is.
Other studies have also highlighted the lack of big data skills as a barrier to organizations capitalizing on the potential of big data. Both of the top responses in Jaspersoft’s survey show this is indeed a problem — especially the top answer since data does not have to be unstructured to be considered big data.
It is also interesting that most of the company’s included in the survey are not dealing with the massive data volumes often discussed by vendors and technology evangelist. Only two percent of survey participants said their project would manage exabytes of data, and just slightly more, eight percent, dealt with petabytes. Most respondents, 78 percent, indicated their projects dealt with terabytes of data or less. Twelve percent of respondents were unsure of their project’s total estimated data volume.
E-commerce, financials and customer relationship management enterprise applications were the biggest source of content for big data projects. Respondent could select more than one response, and 363 responses specified data originated from one of the top three systems. Hadoop may be getting the most press for managing big data, but respondents overwhelming indicated (60 percent of responses) the data for their big data analytics project was stored in a traditional relational database. Only 26 percent of responses mentioned Hadoop HDFS or Hbase.
The Big Picture for Big Data
Jaspersoft’s study supports what many of the other big data studies have shown. The majority of organizations are beginning to invest in big data, but big data skills remain a significant challenge. The technology landscape is still diverse, and relational databases continue to play an important role in managing growing data volumes.