Microsoft Azure Government Gets Azure HDInsight, Cognitive Services and Power BI

Power BI Pro and HDInsight are now part of Azure Government in the U.S., Azure Storage JavaScript client library for browsers and Diagnostic logs in Azure Stream Analytics, now in preview.

Share online:

Microsoft Azure Government in US now includes Power Bi Pro, HD Insight, bringing advanced data capabilities

"These cloud-based data analytics and visualization services will empower government employees to visualize data, glean valuable insights and intelligence from their vast data resources to take informed action to improve citizen services, reduce taxpayer costs and increase efficiency, all while adhering to the most stringent security requirements," writes Curt Kolcun.

Azure HDInsight deploys and provisions clusters from the Apache Hadoop ecosystem (including Spark, Storm, Hbase) in the cloud, providing a software framework designed to manage, analyze, and report on big data.

HD Insight Premium is currently in Preview in public, and is expected in Azure Government in the coming months. Benefits include:

  • Automatic provisioning of Hadoop clusters.
  • High availability and reliability of clusters.
  • Integration with other Azure services, including Web apps and SQL Database.Limitations:
HDInsight and Power BI in Azure Government
HDInsight and Power BI in Azure Government

Currently, there are some limitations as well including: "HDInsight is not available on Windows, and Azure Data Lake Store is not currently available in Azure Government. Azure Blob Storage is the only available storage option at this time," the team writes.

Power BI, on the other hand gives powerful tools to visualize, analyze, and share data. The following features are available in Power BI US Government:

  • Create and view rich dashboards and reports
  • Data capacity limits
  • Scheduled data refresh
  • Refreshable team dashboards
  • Active Directory groups for sharing and managing access control
  • Import data and reports from Excel, CSV, and Power BI Desktop files
  • Data Management Gateway
  • All data is encrypted in both Azure SQL and Blob Storage for Power BI
  • Connect to services with content packs

Check out this demo to see how easy it is to integrate HDInsight on Linux into Government, and how to use Power BI to create a Government-specific data solution:

On a side note, Power BI has received the 2017 Esri Partner Conference – New Technology Integration award for innovations in map visualization and geo-analytics. Below you can see the summary of Power BI Desktop March feature release:

Report view

Report theming preview

  • New matrix visual preview
  • Textbox font color
  • Numeric range slicer preview
  • Percent of total on pie and donut chart data labels
  • Cross-highlight using multiple series

    Analytics

    Clustering is now generally available

    Data connectivity

    New Connector: Azure Analysis Services

  • Azure Data Lake Store connector is now generally available
  • DB2 connector: option to specify package collection
  • Combine binaries: ability to specify sample file to use

    Query editing

    Split column by delimiter: automatic detection of delimiter character

  • In the video below, you can find more about the major updates:

    Also, a preview of Cognitive Services in Azure Government, which enables natural and contextual interaction with tools that augment users' experiences using the power of machine-based intelligence is announced as well.

    "These capabilities are increasingly important to public sector customers, and demonstrate Microsoft's commitment to innovate in the Azure Government Cloud," Kolcun says.

    Newest library Azure Storage Client Library for JavaScript, that enable web development scenarios with Azure Storage is made available today as well along with a preview of Azure Storage JavaScript Client Library for Browsers.

    Azure SQL Data Warehouse training is now available online via the edX training portal to help you learn how to deploy, design, and load data using Microsoft's Azure SQL Data Warehouse, or SQL DW.

    You'll learn about "data distribution, compressed in-memory indexes, PolyBase for Big Data, and elastic scale," the team says.

    Overall score of those who take final exam, will be determined by taking into 30% of accounts grade and combined with weekly quizzes. To pass this course and earn a certificate, you must achieve an overall score of 70% or higher.

    "Azure SQL Data Warehouse (SQL DW) is a SQL-based fully managed, petabyte-scale cloud solution for data warehousing. SQL Data Warehouse is highly elastic, enabling you to provision in minutes and scale capacity in seconds. You can scale compute and storage independently, allowing you to burst compute for complex analytical workloads or scale down your warehouse for archival scenarios, and pay based off what you're using instead of being locked into predefined cluster configurations."

    The course syllabus include:

    • Module 1: Key Concepts of MPP (Massively Parallel Processing) Technology and SQL Data Warehouse
    • This module makes a case for deploying a data warehouse in the cloud, introduces massively parallel processing and explores the components of Azure SQL Data Warehouse.
    • Module 2: Provisioning a SQL Data Warehouse
    • This module introduces the tasks needed to provision Azure SQL Data Warehouse, the tools used to connect to and manage the data warehouse and key querying options.
    • Module 3: Designing Tables and Loading Data
    • This module covers data distribution in an MPP data warehouse, creating tables and loading data.
    • Module 4: Integrating SQL DW in a Big Data Solution
    • This module introduces Polybase to access big data, managing, protecting, and securing your Azure SQL Data Warehouse, and integrating your Azure SQL Data Warehouse into a big data solution.

    Microsoft Azure Storage Explorer 0.8.9, with following recent features:

    • Automatically download the latest version when it is available
    • Create, manage, and promote blob snapshots
    • Sign-in to Sovereigh Clouds like Azure China, Azure Germany and Azure US Government
    • Zoom In, Zoom Out, and Reset Zoom from View menu
    Microsoft Azure Storage Explorer
    Microsoft Azure Storage Explorer

    Diagnostic logs for Azure Stream Analytics, now available in public preview, help examine late or malformed data that cause unexpected behaviors.

    This tool also can remedy errors caused by data that don't conform to query expectations, writes program manager. In addition, diagnostic logs provide rich insights into all operations associated with a streaming job.

    They are turned off by default and can be enabled in "Diagnostic logs" blade under "Monitoring".

    Here are a few examples of data handling errors that diagnostic logs can help with include:

    • Data conversion and serialization errors in cases of schema mismatch.
    • Incompatible types including constraints such as allow null and duplicates.
    • Truncation of strings and issues with precision during conversion.
    • Expression evaluation errors such as divide by zero, overflow etc.

    Microsoft recommends SharePoint Online in Office 365. If that is not the best option for you right now, you should use SharePoint Server 2016. However, building a SharePoint Server 2016 farm in Microsoft Azure infrastructure services requires additional planning considerations and deployment steps.

    Microsoft recommend to use SharePoint Server 2016 High Availability Farm in Azure Deployment Kit, for creating Azure infrastructure and configuring servers of the high availability SharePoint Server 2016 farm. The ZIP file in the TechNet Gallery contains:

    • Microsoft Visio and Microsoft PowerPoint files with the figures for the two dev/test environments and the high-availability deployment
    • All the PowerShell command blocks to create and configure the high availability SharePoint Server 2016 farm in Azure
    • A Microsoft Excel configuration workbook that generates the PowerShell commands to create the SharePoint Server 2016 high availability farm in Azure, based on your custom settings, the team says.
    SharePoint Server 2016 in Azure
    SharePoint Server 2016 in Azure