Data warehouse and analytics professionals under increased pressure - report

Data warehouse and analytics professionals are under increased pressure to develop new data sets faster than ever, according to a new survey from WhereScape.

The data infrastructure automation software provider has released a report in partnership with TDWI, revealing how organisations can accelerate data warehouse development to meet demand for new data sets and insights. The report focuses on how data warehouse professionals can scale to handle continuously growing numbers of data solutions while also delivering each one in record time.

The TDWI Accelerating Data Warehouse Development Pulse Report identifies how analytics professionals are under renewed pressure to develop new data sets and insights - and develop them in shorter time frames than ever.

According to the report, automation and high ease-of-use supported in development tools and data platforms that are purpose-built for data warehousing are key to meeting these growing demands.

The report found the majority of respondents agree agility is needed, in the sense of speed, productivity, flexibility, and innovation (70%), while the vast majority of survey respondents consider DW acceleration important. In fact, almost two thirds (62%) find it extremely important.

According to the research, data warehouse professionals want their data quality issues automatically recognised and remediated (64%), and they want automation for data validation and verification (51%).

DW developers are looking forward to tools supporting intelligent assistants that guide developers (19%), possibly based on artificial intelligence and machine learning, and 33% agree that automation tools are affordable, compared to hiring and consultants.
 
Tools conducive to modern data development methods, particularly, agile, lean, virtual, and other methods known to be fast, flexible, and efficient in rapid prototyping environments are a core area of best practice. The report highlights the importance of a skilled data warehouse team that knows the appropriate methodologies for accelerating data warehouse development. Tools are important, but team competency is an equally critical success factor for accelerating data warehouse development.

In developing a best-practice approach to accelerated data warehouse development, organisations should realise that DW development must keep up with an accelerating and evolving world, the report says.

However, data warehouses should continue to support existing business goals, and while supporting mature goals around business reporting and dashboards, modern data warehouse development should also focus on advanced analytics, big data, and real-time operations.

In addition, accelerating data warehouse deployment also depends on employing accelerated DW methods and automated DW tools, the modernisation of development methods and a clear focus on automation to cover broader data disciplines, enabled by software that supports automation and agility.

"Data warehouse professionals must develop more data solutions than ever, faster than ever," says Rob Mellor, VP - GM, WhereScape EMEA.

"As a result, it is vital that they are given the tools, resources and guidance to develop these best practices so they can balance the need for effective delivery against the growing pressure to accelerate development," he says.

"Organisations that achieve these goals will be better placed to deliver the insight they need to make more effective decisions, maximise efficiency and boost competitive advantage."