VMware Boosts Hybrid-Cloud Visibility, Management

VMware pushed its operations management tool to the cloud with a new software-as-a-service (SaaS) form factor and newly added hybrid cloud support for Google Cloud Platform (GCP) and vSphere Kubernetes deployments in addition to Amazon Web Services (AWS) and Microsoft Azure.

vRealize Operations is VMware’s on-premises software product that provides operations management across applications, infrastructure, and clouds. The new vRealize Operations Cloud, which is generally available today, has the same user interface and features as the on-premises version. “It offers everything you can do with vRealize operations on prem,” said Taruna Gandhi, VP of product marketing for cloud management at VMware. This includes “performance optimization, which includes fully automated workload balancing and placement” based on business and operational intent, she added.

The product also helps enterprises better manage costs and improve efficiency by making recommendations about procurement and rightsizing workloads. Both versions use artificial intelligence (AI) for troubleshooting — predicting and preventing issues across customers’ hybrid and multi-cloud environments, Gandhi said. And it continually scans these environments to enforce IT and regulatory standards around configuration and compliance, and automatically remediates any problems.

‘Major In Hybrid Cloud’

Some of the new features in both the SaaS and on-premises versions include better public cloud cost comparisons via integration with VMware’s CloudHealth and improved VMware Cloud on AWS migration.

CloudHealth is VMware’s public cloud management platform. To explain the relationship between CloudHealth and vRealize Operations, “think of vROps as the defacto solution for VMware hybrid cloud, VMware SDDC and VMware Cloud on AWS, whereas CloudHealth is heavily focused on the public-cloud side,” Gandhi said. “While vROps has a major in hybrid cloud, it has a minor in public cloud.”

The updated software also supports the newly released vSphere 7.0 with Kubernetes and Google Cloud Platform, which VMware formed a hybrid-cloud partnership with last summer.

And while vRealize Operations Cloud became generally available today, 33 customers have been beta testing the SaaS product since February, accounting for 132,585 total virtual machines (VMs). This included 124 different vCenter accounts, and 11 VMware cloud on AWS accounts. Beta customers include German stock exchange Börse Stuttgart, Australian gambling and entertainment group Tabcorp, and Israel-based enterprise software and IT services firm Trilogy.

VMware Customer Traction

Börse Stuttgart, an existing vRealize Operations on-premises customer, has been working with VMware to migrate administrative data to public cloud since 2019. While some highly regulated trading data must remain on-premises, “they are trying to get out of the business of maintaining and managing as much of their on-premises software as they can, Gandhi said. The recent COVID-19 pandemic has amplified this move to the cloud because while stock traders still work the trading floor, “everyone else is working from home. So anything they can do remote, any troubleshooting, really helps them out.”

In a statement, Stephan Wiechert, IT system specialist at Börse Stuttgart, said vRealize Operations Cloud has improved services monitoring for faster troubleshooting. “VMware vRealize Operations Cloud works smoothly,” he said. “I get a message before an upgrade, then it happens automatically. A few minutes later, I get another message saying everything is finished. I’m saving up to 10 hours a month on upgrades and troubleshooting. That’s success to me.”

Meanwhile, Trilogy wasn’t an existing vRealize Operations customer. “They have no physical real estate. They are all in the cloud, mostly on VMware Cloud on AWS and AWS native,” Gandhi said.

Trilogy reduced costs and improved and improved efficiency across its hybrid-cloud environment with vRealize Operations Cloud, said Fuat Ulugay, SaaS operations manager at Trilogy, in a statement.

“With VMware vRealize Operations Cloud we were able to gain full visibility across our environment — which we didn’t have previously,” Ulugay said. “The insight delivered by the service led us to immediately reduce our hosts by 20%, saving us thousands of dollars in the process. In less than a week, we were able to right-size our environment, optimize our performance, and see what we needed for capacity planning.”

https://www.sdxcentral.com/

IoT roundup: Tech companies step up to help fight against COVID-19

We’ve seen few efforts as monumental as the global fight against the COVID-19 pandemic. While the tech sector as a whole has suffered less than many other industries, individual companies and organizations are stepping up to help battle the virus’ spread, and IoT in particular offers numerous methods to help keep people healthy.

In the healthcare sector, a San Antonio-based startup has developed an autonomous robot that can sanitize hospital rooms on its own, obviating the need for human workers to be exposed to the virus. XENEX’s LightStrike robots use powerful UV lights to kill off pathogens lingering on surfaces, and they’re coordinated and monitored in near-real-time via AT&T’s cellular network. The mobile operator also says that it’s partnering with a second company, Brain Corp, to connect a similar type of robot worker, this one designed to clean stores, restock items and even provide automated delivery services.

“The health crisis is a clear example of how advanced technology and IoT solutions can radically improve the way we approach and solve societal challenges,” said AT&T senior vice president Chris Penrose, in a statement.

https://www.idginsiderpro.com/

Cloud-Based Quantum Computing Shows Increasing Enterprise Interest

Over the past year quantum computing has started moving from the realm of computer science and research into something that enterprises might be able to use in the workplace. In fact, the emergence of quantum computing in the cloud has excited curiosity among tech buyers who are now looking at practical applications for this kind of computing.

Quantum Computing Adoption

Tech buyers are not the only ones that are looking at it either. Recent research from IDC points to the fact that behind this interest is the fact that quantum computing offers organizations the possibility of improved AI capabilities, accelerated business intelligence, and increased productivity and efficiency.

The research, entitled Quantum Computing Adoption Trends: 2020 Survey Findings (behind paywall) says that while cloud-based quantum computing is a young market, and allocated funds for quantum computing initiatives are limited (0-2% of IT budgets), end-users are optimistic that early investment will result in a competitive advantage.

There are number of industries that are particularly interested in it, among them manufacturing, financial services and security industries are all experimenting with more potential use cases, developing advanced prototypes, and are further along in their implementation status. However, complex technology, skillset limitations, lack of available resources, and cost is deterring some organizations from investing in quantum computing technology.

The rise of quantum computing has not gone unnoticed by investors who have been sniffing around cutting edge companies that are currently developing new ways to enable enterprise access quantum computing. Take Berkeley Calif.-based Rigetti Computing.Founded in 2013 by Chad Rigetti, it has made its quantum computers available over the cloud since 2017, and recently raised $71 million in a recent funding round for the company’s future development.

Through its Quantum Cloud Services (QCS) platform, its machines can be integrated into any public, private or hybrid cloud. In a recent interview Michael Brett, senior vice president of applications at Rigetti, pointed out that over the past two years the number of companies providing access to quantum computers has gone from two (IBM and Rigetti) to many more.

However, in recent months this has been transformed by the entry of big cloud vendors. Rigetti for example has a deal with AWS to use its cloud compute time. Microsoft Azure offers a similar service, and other cloud players are expected to follow. This not only opens a more efficient array of computing, but it changes the market dynamics: “AWS is both an aggregator and a distributor of quantum compute, as well as a provider of software tools to help developers,” Brett told DigiFin.

Quantum Developments

SRI International (SRI) is an American nonprofit scientific research institute and organization headquartered in Menlo Park, California It is also one of the supporters of the Quantum Economic Development Consortium (QED-C) which is also supported by the National Institute of Standards and Technology (NIST) in the Department of Commerce and by more than 70 U.S. companies from across the supply chain.

Just before Xmas this year QED-C held a workshop to identify cryogenic technology advances that will enable a ten-fold improvement in quantum information science and technology (QIST) in the next decade “Cryogenics is slowing progress toward the establishment of a quantum industry, which will have both economic and national security implications, and which will create benefits to mankind that we can only dream of today,”  Luke Mauritsen, founder and CEO of Montana Instruments said of the project Workshop

As well as outlining the path for future development of quantum computing, it also identified some of the problems that need to be answered before quantum computing can move forward. Among the problems that need to be solved is reductions in size, weight and power requirements that will enable commercial applications that today are not feasible.

On top of that for many at research universities and in small companies, cost of the systems that are required for certain applications, which can be more than $1 million, is a barrier to entry. Finally, the dwindling pipeline of relevantly educated workers is a growing problem for companies in the field.

Quantum Potential

That said, Itamar Sivan CEO of Israel-based Quantum Machines, which also received a recent funding round, this time for $17.5 million, argues that cloud-based quantum computing is a field that holds incredible potential. The advantages seem obvious quantum computing holds a promise for immense computational power, far beyond anything classical computing will ever be able to provide. Like the incredible growth of cloud computing for AI and big-data applications. the anticipated adoption by large corporations of quantum cloud computing, will surely be immense. “So, in theory, as IDC has stated, quantum cloud investment can provide advantages across all industries, and we highly recommend corporations to begin allocating resources to the field,” he said.

Such investments would be gradual, as the field progresses, to allow companies, both, not to be left behind when quantum reaches its tipping point, or even better, to gain a first-movers advantages as soon as this happens. Whether this happens in two or five years, companies which have wisely invested in the field, might benefit from an unprecedented advantage.

Is Quantum Computing Necessary?

But is quantum computing necessary? Jay Valentine of  Austin-based Cloud-Sliver believes not. Instead, he believes that Edge computing will drastically impact quantum. Quantum computing, he said, in the way it is being practiced is a hardware solution that runs very specific apps superfast. It is massively expensive, and the skills needed to make it work are not there. Edge computing (vs computing at the edge) has created an entirely different distributed tech stack that enables any app to be broken up into infinite pieces, run simultaneously on tiny hardware devices like a Raspberry Pi or embedded device. “These today actually exceed the speed of any quantum computer with published results. And it is delivered with traditional hardware, (Raspberry Pi), inexpensively, today, and is in production at several sites” he said.

He said that already, we are taking apps that consume an $87 million data center, for 90 hours to come to a result, and we deliver the same in 20 seconds on a single Raspberry Pi. This kind of EDGE tech will eliminate most of the need for quantum computers.

https://www.cmswire.com/

Australian government readies COVID-19 information app

The federal government is looking to publish an app to help distribute information on the COVID-19 outbreak.

The COVID-19 Gov't Mobile Platform opportunity was posted on the Digital Transformation Agency's (DTA) Digital Marketplace, which is touted by the agency as being a "simple, open platform that brings government buyers and digital sellers together".

"This request seeks to engage an experienced seller to partner with the DTA to continue to develop, support, and host a government mobile platform to allow the dissemination of COVID-19 virus information, related restrictions, and other supporting advice and directions," the overview says.

The opportunity was posted on Wednesday and will remain open until 6 pm AEDT on Thursday.

So far, only one vendor has been invited to participate.

The length of the contract is six months, with the option for a further six-month extension.

Essential criteria is listed as experience in developing mobile platforms, and desirable criteria is that the successful vendor has the ability to support and host mobile platforms during a pandemic.

The app was initially announced when the federal government first unveiled its COVID-19 support package. It falls under the new national communications campaign, which will see AU$30 million spent on providing people with "practical advice on how they can play their part in containing the virus and staying healthy", such as through the app.

The Australian government on Wednesday began its text message campaign, telling nearly 36 million mobile numbers how to navigate the health of individuals and the broader community.

"As the spread of the coronavirus increases, it's vital every Australian understands the practical action they must take to look after themselves and help us protect those most at risk," a statement from Minister for Health Greg Hunt, Australian Chief Medical Officer Professor Brendan Murphy, and Minister for Communications, Cyber Safety and the Arts Paul Fletcher said.

The government said it would continue to use text messages as one of its communication methods.

This follows Canberra suffering backlash earlier this week for its inability to provide appropriate tech capability to handle the number of Australians seeking government assistance.

In the wake of COVID-19, social distancing measures and business closures have left many without a job. In response, the Australian government announced, over the past fortnight, a handful of measures to support the newly unemployed as more than 1 million Australians could be forced onto welfare.

However, on Monday, thousands were unable to access the government's myGov online portal to sign up for income assistance.

"We are deeply sorry about this," Morrison said on Tuesday night.

"We've gone from 6,000 to 50,000 to 150,000 all in the space of, a matter of a day. And tonight, they're working to boost it again. I would say to Australians, yes, we are terribly sorry, but at the same time, we are asking Australians, even in these most difficult of circumstances, to be patient. Everyone is doing their best."

OPEN-SOURCING TRACING TECH IN SINGAPORE

Earlier this week, Singapore announced it will open-source its COVID-19 contact-tracing app, TraceTogether.

"GovTech Singapore is now working around the clock to finalise our protocol reference documents and reference implementation, so that others may deploy their own flavours of TraceTogether -- each implementing the BlueTrace protocol," Minister-in-charge of the Smart Nation Programme Office initiative Vivian Balakrishnan said in a Facebook post.

"We believe that making our code available to the world will enhance trust and collaboration in dealing with a global threat that does not respect boundaries, political systems or economies."

TraceTogether is built on the BlueTrace protocol, designed by the Government Digital Services team at Government Technology Agency of Singapore.

Participating devices exchange proximity information whenever an app detects another device with the TraceTogether app installed.

Balakrishnan said TraceTogether has been installed by more than 620,000 people.

The app uses the Bluetooth Relative Signal Strength Indicator (RSSI) readings between devices across time, to approximate the proximity and duration of an encounter between two users, the TraceTogether website explains.

"This proximity and duration information is stored on one's phone for 21 days on a rolling basis -- anything beyond that would be deleted. No location data is collected," it adds.

"If a person unfortunately falls ill with COVID-19, the Ministry of Health (MOH) would work with them to map out their activity for past 14 days, for contact tracing. And if the person has the TraceTogether app installed, they can grant MOH to access their TraceTogether Bluetooth proximity data -- making it quicker for MOH to contact people who had close contact with the infected individual, to provide timely guidance and care."

https://www.zdnet.com/

COVID-19 slams tech outfits and startups in India

With COVID-19 cutting a devastating swath throughout the world, what everyone wants to know about India is how bad the situation really is. In a country with a large but mostly poor population of 1.3 billion and a per capita of around just $2,000, a virus such as this can spread like wildfire and cause devastation. 

So far, India has seen 612 cases and twelve deaths, but this is easily a questionable number considering the lack of testing kits, testers, and the country's massive population. China shut Wuhan down almost instantly and still suffered. India, like China, could also be deeply affected, especially if it has under-reported its figures. Realising this, the Indian government has done the smart thing by implementing a 21-day lockdown -- or de-facto "house arrest" -- along with an international and domestic flight ban, and a stoppage of the railway service. 

The Indian government, with help from tech firm Haptik, has also launched a WhatsApp chatbot called MyGov Corona Helpdesk where people can text with questions about the virus. In turn, they can receive instant responses, including information about where they could receive assistance. 

With government implementing these initiatives, its hope is that by the end of this period, much of the immediate threats from the virus will have blown over.  

As the body continues to rise then, it may seem trivial to talk about the state of the tech sector as I am going to try and do, but the reality is that tech employs a wide swath of people, including Uber and Ola drivers and delivery personnel. With potentially nothing in their bank accounts to rescue them, their livelihoods are directly at stake with business closures arising from COVID-19.

Here's a short account of how various parts of the tech sector have fared with the ongoing crisis.

ONLINE GROCERIES

A few days ago, I wrote an article about how edtech firms in India are experiencing an upswing with schools being shut down during the exam period. Similarly, the revenues of internet-enabled grocery outfits such as Grofers and Bigbasket have gone through the roof, growing by double according to Quartz, as Indians go through a surge of panic buying. Average basket values have also gone up by as much as 20%. 

Experts say this will continue to have stickiness as new users discover the convenience of online shopping where they would not have otherwise. However, online shopping has taken a big hit in general, with Amazon to stop shipping non-essential products.

STARTUPS

For many tech companies that are still in their infancy, this pandemic will probably come as a bitter blow. February already registered a 19-month low for investments and although this was primarily due to fewer big-ticket acquisitions, it is no doubt be a harbinger for things to come. 

Sequoia recently sent out an email to its companies to "question every assumption" about their business and to start thinking about how to cut spending and jobs. Firms that have focused mainly on customer acquisition rather than generating profits will also find that the game has dramatically changed. According to Livemint, fundraising has already ground to a halt, with many companies only having control over their burn rate.

One promising startup sector that has already been hard hit is logistics, which had been booming until now. Shipsy, a Gurugram-based company that uses a digital platform to connect exporters and importers, has already seen their business plummet 25%, and with global shipping and transport having ground to a halt, this looks like the tip of the iceberg.

ENTERTAINMENT

Just like it did in Europe a few days ago, Netflix announced that it would throttle its traffic over Indian telecom networks by 25% in order to alleviate the data congestion that has arisen from cooped up people binge-watching their content. Youtube and Amazon have also made similar announcements.

MANUFACTURING

One of the biggest potential impacts to India's employment and economy could be the shuttering of manufacturing plants and assembly lines in the tech sector. For example, Xiaomi has eight factories that churn out smartphones, smart TVs, and power banks. One can only imagine the ripple effect that this would have on revenues, handset supply, and jobs if it were forced to close the doors on all of its factories.

So far, Xiaomi, Lenovo-Motorola, and Lava have already been impacted, with some of their smartphone factories being forced to shut down following a diktat by state government. 

ONLINE TRAVEL

Needless to say, no one is even thinking of travelling so you can imagine the implosion that is currently taking place for online-enabled homestays, hotels, and travel websites like the rapidly growing Oyo. Leading travel aggregator Yatra.com said that 35% of bookings that include travel and hotel to international destinations have already been cancelled. 

https://www.zdnet.com/

Hybrid-cloud management requires new tools, skills

Hybrid cloud environments can deliver an array of benefits, but in many enterprises, they're becoming increasingly complex and difficult to manage. To cope, adopters typically turn to some type of management software. What soon becomes apparent, however, is that hybrid cloud management tools can be as complex and confounding as the environments they're designed to support.

A hybrid cloud typically includes a mix of computing, storage and other services. The environment is formed by a combination of on-premises infrastructure resources, private cloud services, and one or more public cloud offerings, such as Amazon Web Services (AWS) or Microsoft Azure, as well as orchestration among the various platforms.

Any organization contemplating a hybrid cloud deployment should begin building a transition framework at the earliest possible stage. "The biggest decision is what data and which applications should be on-premises due to the sensitivity of data, and what goes into the cloud," says Umesh Padval, a partner at venture capital firm Thomvest Ventures.

Numerous other issues also need to be sorted out at the start, including the ultimate destination of lower priority, yet still critical, data and applications. Will they be kept on premises forever or migrated at some point into the cloud? With applications and data scattered, security is another major concern. Operational factors and costs also need to be addressed at the very beginning. "Your email application may run great in your data center, but may operate differently in the cloud," Padval notes.

Hybrid cloud tools immature yet evolving

A complex hybrid cloud requires constant oversight as well as a way to intuitively and effectively manage an array of operations, including network performance, workload management, security and cost control. Not surprisingly, given the large number of management tasks needed to run an efficient and reliable hybrid cloud environment, adopters can select from a rapidly growing array of management tools.

"There’s a dizzying array of options from vendors, and it can be difficult to sort through them all," says R. Leigh Henning, principal network architect for data center operator Markley Group. "Vendors don’t always do the best job at making their differentiators clear, and a lot of time and effort is wasted as a result of this confusion. Companies are getting bogged down in an opaque field of choices."

The current hybrid cloud management market is both immature and evolving, declares Paul Miller, vice president of hybrid cloud at Hewlett Packard Enterprise. Vendors are still getting a handle on the types of management tools their customers need. "Offerings are limited and may not be supported across all public, on-premises and edges," Miller adds.

Perhaps the biggest challenge to hybrid cloud management is that the technology adds new, complex and frequently discordant layers to operations management. "Many solutions have compatibility restrictions on the components they can manage, locking your management platform into a vendor or group of vendors, which may or may not align with your current or future system architecture," warns George Burns III, senior consultant of cloud operations for IT professional services firm SPR.

A lack of standardized APIs, which in turn results in a shortage of standardized management tools, presents another adoption challenge. "The lack of standardized tools increases operational complexity through the creation of multiple incongruent tools; this leads to vendor lock-in and, in some cases, gross inefficiencies in terms of resource utilization," explains Vipin Jain, CTO of Pensando, a software-defined services platform developer. "To make it worse, these kinds of problems are typically 'solved' by adding another layer of software, which further increases complexity, reduces debuggability, and results in suboptimal use of features and resources."

Meanwhile, using standardized open-source tools can be an effective starting point to safeguard against compatibility issues. "Cloud Native Computing Foundation (CNCF) tools, such as Kubernetes and Prometheus, are good examples," Jain says. "Open-source tools from HashiCorp, such as Vault, Vagrant, Packer, and Terraform, [provide] a good normalization layer for multi-cloud and hybrid cloud deployments, but they are by no means sufficient," he notes. Ideally, the leading public cloud vendors would all agree on a standardized set of APIs that the rest of the industry could then follow. "Standardization can be a moving target, but it's critical from an efficiency and customer satisfaction perspective," Jain says.

Developers writing API configurations, as well as developers using API configurations, form a symbiotic relationship that should be mutually maintained, Burns advises. "Hardware vendors need to be open about changes and enhancements coming to their products and how that will affect their APIs," he explains. "Equally, management platform developers need to be mindful of changes to hardware platform APIs, [and] regularly participate in testing releases and provide adequate feedback to the vendor about results and functionality."

Prioritize management requirements; expect gaps

Even when everything works right, there are often gaps remaining between intended and actual management functionality. "In an ideal world, developers would have the perfect lab environments that would allow them to successfully test each product implementation, allowing functionality to be seamless across upgrades," Burns observes. "Unfortunately, we can’t expect everything to function perfectly and cannot forgo [on-site] testing."

When selecting a hybrid cloud management platform, it's important to not only be aware of its documented limitations, but also to know that nothing is certain until it's tested in its user's own hybrid cloud environment, Burns advises. "Gaps will exist, but it's ultimately your responsibility to fully identify and verify those gaps in your own environment," he says.

Further muddling the situation is the fact that many management tool packages are designed to supply multiple functions, which can make product selection difficult and confusing. "To simplify, customers need to consider which features are most important to them based on their use cases and can show a quick return on investment, mapping to their specific cloud journey," Miller explains.

Real-world experience with hybrid cloud management

Despite management challenges, most hybrid cloud adopters find a way to get their environment to function effectively, reliably and securely.

Gavin Burris, senior project leader, research computing, at the Wharton School of the University of Pennsylvania, appreciates the flexibility a hybrid cloud provides. "We have a small cluster ... that's generally available to all the faculty and PhD students," he notes. The school's hybrid environment supports a fair share prioritization scheme, which ensures that all users have access to the resources they need to support their work. "When they need more, they're able to request their own dedicated job queue that's run in the cloud," he says.

Burris, who uses Univa management products, says that having a management tool that allows fast and easy changes is perfect for individuals who like to maintain firm control over their hybrid environment. "I like to do things with scripting and automation, so to be able to go in and write my own rules and policies and build my own cluster with these management tools is really what I’m looking for," he explains.

James McGibney, senior director of cybersecurity and compliance at Rosendin Electric, an electrical contractor headquartered in San Jose, Calif., relies on a hybrid cloud to support a variety of essential operations. "Approximately two years ago we embarked on our journey from an on-premises disaster recovery, quality assurance and production environment to a cloud migration encompassing hundreds of terabytes of data," he says. McGibney relies on a management console provided by AWS and VMWare. The tool meets his current needs, but like many hybrid cloud administrators, he's keeping a close eye on industry developments. "We're currently investigating [other] options, just to see what’ out there," he says. Yet he doesn't expect to make any changes in the short term. "We're happy with the tools currently provided by AWS and VMware."

Sharpen network skills for hybrid cloud

Selecting a hybrid cloud management platform is not as simple as purchasing software and spinning up some VMs to run it. "During implementation, ensure that you have selected the proper product owners and engineers, and then determine what, if any, additional education or credentials they will need to effectively deploy and maintain the platform," Burns suggests. "Fully define your architecture, ensure buy-in from your staff, work with them to identify education gaps and create a solid operational plan for going forward."

Most hybrid cloud management tasks focus on configuration and access control operations, which tend to be both complex and challenging to implement. "At the same time, the beauty of the cloud is its ability to automate," says Mike Lamberg vice president and CISO at ION Group and its Openlink unit, which provides risk management, operations and finance software. Yet deploying a high level of automation also requires new skills and developers who can expertly handle the demands of virtual software-defined infrastructures as well as traditional environments. "We can’t assume that because teams can build applications in physical data centers that these skills will translate as they move to the cloud; new skills are required for success," Lamberg notes.

Hybrid cloud management requires a new team mindset. "IT networking staff literally need to unlearn what they know about physical networks and connectivity and recognize that the moving of packets and data is now handled by a forwarding software configuration, not by physical routers or switches," Lamberg says. "You can’t take what you did in building and supporting physical data centers and just apply it to the cloud—it simply doesn’t work."

In the big picture, transitioning to a hybrid cloud environment can solve many problems, yet it can also create some new obstacles if not properly implemented and managed. "Don't rush into any decision without considering all the points of impact that you can identify," Burns advises. "Make sure that you understand the breadth of a hybrid infrastructure and how it will be used to address business needs."

https://www.networkworld.com/

How to query and extract data from SaaS applications

Behind every SaaS application are databases storing business information about employees, suppliers, customers, and other partners. SaaS applications support workflows such as CRM for sales and marketing, cloud ERPs for financials, workforce management for human resource functions, and other enterprise and departmental services. Today, many businesses use a wide range of SaaS applications—from mainstream products such as Salesforce, Slack, Workday, and Atlassian, to many smaller SaaS tools.

SaaS applications shouldn’t operate in silos, and most organizations need to integrate capabilities across them and with other enterprise applications managed in private or public clouds.

If a workflow across multiple applications requires application integration, then development teams can leverage a SaaS platform’s APIs to trigger events from one platform to another. Enterprise integration platforms such as Boomi, SnapLogic, or MuleSoft are options when many applications and services need integration. If lighterweight integrations that follow the If This, Then That pattern are required, then an IFTTT platform may provide sufficient integration. Development teams should also explore low-code platforms such as Appian, OutSystems, and PowWow if they are developing new applications that connect to multiple SaaS and enterprise workflows.

Leveraging SaaS data for different business needs

What if you need to integrate the data from a SaaS platform with other data sources? There are a few reasons why data integration across SaaS tools may be required:

• Business analysts want to develop reports and dashboards using this data.
• Data science teams want the data for machine learning experiments.
• Business teams want to centralize the data to support workflows and other types of applications. For example, marketing teams often use customer data platforms or master data platforms to centralize data on customers, products, and other business entities.
• IT teams should extract the data for backups or enable transitioning data to other platforms.
• Legal teams sometimes need to perform legal discovery on the underlying data.
• Data stewards often want to cleanse, transform, or enrich the underlying data.

Sure, you can leverage the SaaS platforms’ APIs to extract data, but this may require a significant development effort to learn the APIs, understand the SaaS platform’s data model, create data stores for any new data, write the code to load the data, and develop the logic for any transformations. In addition, IT teams have to define cloud or data center infrastructure to host this application or service. Lastly, ongoing support is required for any data integrations designed to run on a schedule or on demand. Developing the integration from scratch may be expensive for development teams and IT organizations with other, more strategic priorities.

Another methodology is to consider data integration, data streaming, ETL (extraction, transformation, and loading), or other data prep platforms. Using a data integration platform may be the optimal method when working with large volumes of data that frequently change, since these platforms enable flexible extraction and transformation. However, they also require upfront development for the integration before end-users get access and utilize the information.

Lighterweight means of querying and managing SaaS data may be desirable. Sometimes, these are useful to experiment, discover, and prototype rapidly. Other times these approaches can easily be used for operation or production needs, especially when data volumes are low and query throughput isn’t significant. Here are three options.

1. BI platforms that directly query SaaS applications

If your primary requirement is reporting, then many self-service BI and data visualization platforms have direct connectors to the more popular SaaS applications.

• Tableau can connect to platforms such as Intuit Quickbooks, Google Analytics, LinkedIn Sales Navigator, ServiceNow, Eloqua, Marketo, and Salesforce.
• Microsoft Power BI also integrates with online services such as Adobe Analytics, Facebook, GitHub, MailChimp, Stripe, Quick Base, and Zendesk.
• Domo claims to have more than a thousand connectors, including platforms such as HubSpot, Jira, Instagram, Qualtrics, Shopify, SurveyMonkey, Twitter, and Workday.

At a minimum, these integrations provide an easy way to query and discover the underlying SaaS data sources. At best, the out-of-the-box integration is sufficient for end-users to create the required data blending, reports, and dashboards.

There are some considerations.

• These platforms enable joins and data blends when columns have matching keys. They become harder to use if significant data transformation is required before integrating the data source or blending it with other data sources.
• Review whether SaaS data integrations are performed with real-time queries, or whether the data is extracted or cached.
• Performance may be a factor if the SaaS application contains large data volumes, if there are complex joins with many other data sources, or if dashboards will be utilized concurrently by many users.

2. Platforms that emulate ODBC, JDBC, OData, or other drivers

If the business needs to go beyond reporting and dashboarding, and a lightweight integration approach is still desirable, then some commercial tools convert SaaS APIs into standard database drivers such as ODBC, JDBC, or OData. Two options for drivers to common SaaS platforms are Progress DataDirect and CData Driver Technologies.

The driver method may be most useful to data science teams who want to perform ad hoc queries into SaaS databases before pulling the data into their analysis. It’s also a good option for application developers who require real-time querying of SaaS application data.

Development and data science teams should investigate the performance of this integration, especially if high query volumes, large data sets, or low latency is required. Also, many SaaS applications throttle or charge customers based on API usage, so this may be a factor if higher query or data volumes are needed.

3. Lightweight ETL platforms that sync SaaS data to cloud databases

One final idea is to instrument a data integration out of the SaaS application into a cloud database that your organization sets up and manages. This strategy adds some operational complexity and costs, and it may not be ideal if real-time querying of the SaaS application data is required. But it does have several advantages:

• It provides more control over the database platform and data architecture that business users, data scientists (including citizen data scientists), and application developers utilize. The platform and architecture should meet the volume, performance, and latency requirements.
• Storing the data independent of the SaaS database provides greater flexibility to transform, join, cleanse, cube, or aggregate data as required by downstream users and applications.
• If data security, data privacy, or other data governance controls for querying this data are different from the access and entitlement controls available in the SaaS applications, then hosting the data in a separate database may be required.
• Hosting the data independent of the SaaS platform may be more cost-effective for higher data and query volume needs.

Although you could instrument this integration with data integration or data prep platforms, there are SaaS data integration platforms with out-of-the-box connectors to many SaaS applications. Stitch, a Talend company, is a plug-and-play solution if your objective is to stream data from SaaS applications to cloud databases. You can select what data to replicate and the replication frequency, but it does not provide any tools for transforming or filtering the data. Skyvia offers a similar product, and both have free tiers to let development teams try out integrations. Alooma, part of Google Cloud, focuses on moving data into big data platforms such as Google BigQuery, Amazon Redshift, and Snowflake, and provides some data transformation capabilities.

If your organization is utilizing many SaaS platforms, then a one-size-fits-all strategy may not work. Each integration path supports different SaaS integrations, and the type of integration must align with anticipated business needs. Reviewing the tools and considering multiple options is a best practice, especially when data integration needs vary.

https://www.infoworld.com/