DataCenterNews Asia logo
Specialist data center news for Asia
Story image

No ETL? No worries, with SQL Server 2019's new data virtualisation

By Contributor
Fri 5 Apr 2019
FYI, this story is more than a year old

Article by IT journalist David Williams

Big data is big business, and global software titan Microsoft is making big data its business to streamline businesses with the next release of SQL Server 2019 late this year.

Microsoft states SQL Server 2019’s focus is to make managing a big data environment a lot easier.
In fact, the product supports a rich variety of environments like Hadoop Distributed File System (HDFS), Spark, analytics tools, and even Kubernetes-managed Linux containers.

Previously, Microsoft allowed T-SQL queries inside SQL Server 2016 to pull data from Hadoop and return it in a structured way without moving or copying the data.

In the 2019 release, this concept of data virtualisation is widely expanded to data sources including Oracle, MongoDB, even CSV files.

Data virtualisation means database administrators can virtualise external data in a SQL Server instance, regardless of source, location, and format, so it can be queried like any other set of tables within your SQL Server instance.

Data virtualisation helps you create a single “virtual” layer of data from these disparate sources providing unified data services to support multiple applications and users.

This sounds like a data lake, but unlike a typical data lake there is no need to move data from where it lives when using data virtualisation, and this is a huge advantage over traditional ETL (extract, transform, load) processes that by necessity require delays, extra storage, additional security, and an amount of engineering to setup and maintain.

For database administrators, software developers, data scientists and others this new capability is transformative and powerful in its simplicity.

Quite literally, Microsoft has enabled SQL Server 2019 databases to hold virtual tables which come from some other place on the Internet - whether it is a SQL Server database or otherwise, whether it’s local or far - and query and join and work with these tables just as if they were native to a company’s database.

All complex ETL processes are gone.

There are numerous situations where this delivers tremendous time and functionality savings.

For instance, if a user makes a dashboard showing their company’s revenues by period, office and client out of their main line-of-business app’s database.

Management then asks for budgeted sales figures to be included so the dashboards contrast actuals against the budget.

However, these aren’t included in the main app but instead held in an Excel spreadsheet.

Traditionally, the solution is to manually or periodically upload the spreadsheet into a new table created for this purpose, which your dashboard then queries against.

Or, if they’re using SQL Server 2019, the user merely saves the budget figures as a CSV file on the Finance team’s file share and add this as a virtual table in their app’s database.

The budget figures appear, and can be queried against, and reported on, as if they were always in the database.

Another case: the company needs sales figures to move from its main application into the back-office ERP platform.

Typically, this is achieved by exporting and importing the data between the disparate products.

Or, if someone is using SQL Server 2019, they include relevant tables from the ERP system as virtual tables in the main application’s database, and directly move data via a scheduled stored procedure.

The ERP doesn’t even have to be an SQL Server database, so long as it is accessible.

SQL Server 2019’s data virtualisation supports Cosmos DB, SQL Server, Azure SQL, Oracle, HDFS and DB2.

Other SQL Server 2019 features include

  • transformational insights over structured and unstructured data supporting Hadoop and Spark
  • scalable compute and storage clusters
  • a complete AI platform to train and operationalise R and Python models in SQL Server Machine Learning Services or Spark ML using Azure Data Studio notebooks
  • the choice of programming language and platforms like .NET, PHP, Node.JS, Java, Python, Ruby and more, and deploy the application on Windows, Linux or containers both on-premise and in the cloud
  • real-time analytics on operational data using Hybrid Transactional and Analytical Processing (HTAP)
  • intelligent query processing features that improve scaling of queries
  • security enhancements including Always Encrypted secure enclaves, and Data Discovery and Classification labelling for GDPR
Related stories
Top stories
Story image
Employment
Tech job moves - Forcepoint, Malwarebytes, SolarWinds & VMware
We round up all job appointments from May 13-20, 2022, in one place to keep you updated with the latest from across the tech industries.
Story image
Sustainability
Intel unveils new investments for data center sustainability
Intel has announced two new investments, continuing its efforts to create more sustainable data center technology.
Story image
Digital Transformation
The Huawei APAC conference kicks off with digital transformation
More than 1500 people from across APAC have gathered for the Huawei APAC Digital Innovation Congress to explore the future of digital innovation.
Story image
Sustainability
Legrand unveils Nexpand, a data center cabinet platform
Legrand has unveiled a new data center cabinet platform, Nexpand, to offer the necessary scalability and future-proof architecture for digital transformation.
Story image
APAC
Odaseva expands in APAC and UK with more security features
Odaseva, a data platform for Salesforce, is establishing new headquarters in London as well as a new data center in India.
Story image
Research
New strategies for cloud-native attacks - Aqua Security
New research from Aqua Security reveals attackers are using more sophisticated techniques to target cloud-native environments.
Story image
Digital Transformation
EdgeConneX enters Indonesia, plans for data center campus
EdgeConnex has announced it is expanding its presence in Asia with the acquisition of GTN Data Center in Indonesia.
Story image
Storage
Energy storage demand momentum continues, says BYD
BYD has announced an expansion of its production capacities and will deliver 250,000 units of its energy storage system, BYD Battery-Box Premium.
Story image
Digital Transformation
Multiplex, NEXTDC making strong progress on S3 data centre
Multiplex has made a significant achievement on Stage 1 of NEXTDC’s S3 data centre, ‘topping out’ the structure in the Artarmon on Sydney’s lower North Shore.
Story image
Databricks
Databricks grows in APAC market, expands into Korea
Databricks officially launches a local office in Seoul, Korea, building on existing partnerships with Cloocus, Megazone and the Weverse Company
Story image
Surveillance
Genetec launches new enclosure management system for data centers
Genetec has released a new enclosure management solution that will give data centers the ability to secure, monitor and manage access to racks and cabinets remotely.
Story image
Sustainability
AyalaLand and FLOW partner for data center development
AyalaLand Logistics Holdings Corp (ALLHC) and FLOW Digital Infrastructure have entered into a framework agreement to bolster the development of carrier-neutral data centers in the Philippines.
Story image
Cloud
Colt connectivity with AWS increases services in Asia
Colt Technology Services expands cloud connectivity to AWS Direct Connect Hosted services, with speeds of up to 10 Gbps in Asia.
Story image
Sustainability
NTT launches IoT Services for Sustainability offering
"We know what actions are needed to build a more sustainable future and have a robust suite of technologies available to help deliver this impact."
Story image
Power / Energy
Keysight Technologies introduces new next-gen DPT solution
Keysight Technologies has announced its new next-generation Double-Pulse Tester (DPT) with the PD1550A Advanced Dynamic Power Device Analyser.
Exabeam
Find out how a behavioural analytics-driven approach can transform security operations with the new Exabeam commissioned Forrester study.
Link image
Story image
Sustainability
Siemens showcases new automated solutions for data centers
Siemens has implemented new automated solutions and AI in the Baltic region's largest data center, providing insight into the future of data center management.
Story image
Microsoft
Microsoft unveils adaptive accessories for disability access
Microsoft is introducing an expansive Inclusive Tech Lab to give people with disabilities greater access to technology through new software features and adaptive accessories.
Story image
Data Center
Preventing downtime costs and damage with Distributed Infrastructure Management
Distributed Infrastructure Management (DIM) can often be a lifeline for many enterprises that work with highly critical ICT infrastructure and power sources.
Story image
Data Center
Tier III Ready Datacenter solutions shortlisted for major awards
"These designs will accelerate data center clients' own Tier III certification, reduce the cost, and fast-track their time to market."
Story image
Data Center
CBRE finds record levels of investment in APAC data centers
CBRE's new report finds direct investment in the sector more than doubled in 2021, surpassing investment volumes for the past four years combined
Story image
Hyperscale
SpaceDC partners with Aofei for data center sales in Asia
SpaceDC has partnered with Aofei Data International to sell Aofei's data centers, CDN and SDN in China.
Story image
Tech Data
Tech Data to use Pluribus Networks’ cloud solutions in APAC
Tech Data says using Pluribus Networks' Unified Cloud Fabric solution will be a "game-changer" for its data center infrastructure customers and partners.
Story image
BitTitan
Why tenant consolidation is critical to cloud success
Consolidating tenants can improve cost management, security and engagement after a flurry of reactive activity following the widespread shift to remote operations.
Story image
Talend
Talend introduces new data health solutions for businesses
Talend has announced its latest version of Talend Data Fabric, with the release of Talend Trust Score enabling data teams to establish a foundation for data health.
Story image
Data Center
Sime Darby Berhad to use Equinix APAC data centers
Equinix has expanded its digital infrastructure services, including its International Business Exchange data centers, to Sime Darby Berhad
Story image
SD-WAN
Orange moves Siemens AG’s entire operations to a SD-WAN
Orange Business Services has migrated Siemens AG's entire global operations, 1168 sites across 94 countries, to a SD-WAN
Story image
Sustainability
Daikin and SP Group to build new energy efficient district cooling system
The project, set to be complete by 2025, will create a system with a cooling capacity of up to 36,000 refrigerant tonnes (RT). 
Story image
Power / Energy
DigitalBridge makes $30 million equity investment in LEDC
Leading Edge Data Centres (LEDC) has announced it has secured an AUD$30 million equity investment in its regional edge network from an affiliate of DigitalBridge Group, DigitalBridge.
Story image
Cybersecurity
The 'A-B-C' of effective application security
Software applications have been a key tool for businesses for decades, but the way they are designed and operated has changed during the past few years.
Story image
Colocation
Digital Edge chooses Nortek’s StatePoint for new data center
Digital Edge will use Nortek's StatePoint liquid cooling technology in its new data center, the first commercial colocation operator in Asia to do so.
Story image
Sisense
Data and analytics could be key to higher selling prices in APAC
Sisense's latest report has found that almost half of data professionals in APAC think customised data and analytics can create better selling prices for their products.
Story image
Red Hat
Red Hat expands capabilities to provide streamlined application development in cloud
"Application development is undergoing significant change and developers need tools to support this transformation."
Story image
Sustainability
ABB unlocks sustainable energy upgrades for data centers
ABB says its new microgrid solutions will get data centers ready for the green revolution and use their own energy sources with a reduced carbon footprint.
Story image
Akamai
Akamai announces new products across security, computing
Akamai has announced a series of new products and updates to existing products across its security and compute product lines, including its entry into the infrastructure as a service (IaaS) market.
Story image
Data Center
Equinix enters Africa, closing US$320 million acquisition of MainOne
The completion of the acquisition augments Equinix's long-term strategy to become a leading African carrier-neutral digital infrastructure company.
Story image
Disaster Recovery
Kacific launches emergency connectivity offering, CommsBox
Kacific has announced the release of a new emergency connectivity offering designed to rapidly provide broadband service in emergency or disaster zones.
Story image
Infrastructure
Report - Data investment the key to better business growth
New research from Digital Realty has revealed that almost half (47%) of IT leaders globally believe their business investment in data systems and infrastructure is a key obstacle or concern.
Story image
Sustainability
YTL unveils development of solar-powered data center campus
YTL Power (YTL) has announced the development of a 500MW data center campus in Johor, the first data center park in Malaysia to be powered by solar energy.
Story image
Sustainability
AirTrunk boosts Japan presence with West Tokyo data center
AirTrunk is planning to build TOK2, a new hyperscale data center in Japan which will strengthen the company’s presence in the country.
Story image
Telstra
Telstra expands business offerings in the Philippines
The expansion aims to offer more choice for customers and enhance connectivity into the Philippines, and within the country.
Story image
Cybersecurity
A10 Networks finds over 15 million DDoS weapons in 2021
A10 Networks notes that in the 2H 2021 reporting period, its security research team tracked more than 15.4 million Distributed Denial-of-Service (DDoS) weapons.
Story image
Sustainability
RDA and MVGX partner for sustainable data center development
Red Dot Analytics (RDA) and MetaVerse Green Exchange (MVGX) have entered a strategic partnership to make Singapore's data center development and operations more sustainable.
Story image
Data Center
Digital Edge to build South Korea's largest commercial data center
The project will be the largest commercial data center project in South Korea with total IT power of 120MW and a capital investment of more than KWR$1 trillion.