Upload Documents to Azure Data Lake and Export Data using SSIS

 Introduction

Azure is growing every day. Microsoft created Azure, which is a Cloud Computing service released on 2010.

According to Microsoft, 80% of the fortune 500 companies are using Azure. Also, 40% of the Azure Revenue comes from Startups and independent software vendors. 33% of the Azure Virtual Machines are using Linux. Microsoft expects to earn $20 billion in 2018.

That is why companies are migrating part of the data to Azure and sometimes all the data.

Azure Data Lake is a special storage to analyze Big Data in parallel in Azure. It is optimized for analytics. You can store Social network data, emails, documents, sensor information, geographical information and more.

(more…)

Working with SQL Server in Hybrid Cloud Scenarios. Part 2

As a rule, impersonal information is stored in a public cloud, and the personalized part – in a private cloud. The question thus arises – how to combine both parts to return a single result at a user’s request? Suppose there is a table of customers divided vertically. The depersonalized columns were included in the table located in Windows Azure SQL Database, and columns with sensitive information (e.g., full name) remained in the local SQL Server. Both tables must be linked by the CustomerID key. Because they are located in different databases on different servers, the JOIN statement will not work. As a possible solution, we have considered the scenario, when the linkage was implemented on the local SQL Server. It served as a kind of entry point for the applications, and the cloud-based SQL Server was set up on it as a linked server. In this article, we will consider the case when both, the local and cloud servers, are equal in terms of the application, and the data merging occurs directly in it, i.e. at the business logic level.

(more…)

Working with SQL Server in Hybrid Cloud Scenarios

A hybrid cloud is a fairly attractive model when implementing cloud computing in enterprise information systems since this approach combines the advantages of public and private clouds. On the one hand, it is possible to flexibly attract external resources when needed and reduce infrastructure costs. On the other hand, full control over data and applications that the enterprise does not want to outsource remains. However, in such a scenario, we inevitably face the task of integrating data from various sources. Suppose there is a table with customers, which is vertically divided into two parts. The depersonalized part was allocated in a public cloud, and the information personalizing the customers remained in a local database. For holistic processing inside the application, you need to combine both parts by CustomerID. There are various ways to do this. Conventionally, they can be divided into two large categories: data aggregation at the on-premise database server level which, in this case, will be a single sign on for accessing local and remote data, and data aggregation within the business logic. This article will consider the first approach.

(more…)

Azure SSAS. Cubes in Clouds

Autumn of 2016 was full of events from Microsoft dedicated to analytics. In practice, the company began aggressive promotion of its predictive analytics and BI platform in the cloud. Finally, it has happened – Microsoft has released the preview of Azure SSAS Tabular. In fact, the company has transferred SSAS Tabular to the cloud. Therefore, SQL Server Analysis Enterprise Edition in cloud  supports DirectQuery, partitions, row-level security, bi-directional relationships, and has compatibility level 1200.

(more…)