<?xml version="1.0" encoding="UTF-8"?><rss version="2.0"
xmlns:content="http://purl.org/rss/1.0/modules/content/"
xmlns:wfw="http://wellformedweb.org/CommentAPI/"
xmlns:dc="http://purl.org/dc/elements/1.1/"
xmlns:atom="http://www.w3.org/2005/Atom"
xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
>
<channel>
<title>SQL Server Blog</title>
<atom:link href="https://blogs.technet.microsoft.com/dataplatforminsider/feed/" rel="self" type="application/rss+xml" />
<link>https://blogs.technet.microsoft.com/dataplatforminsider</link>
<description>Official News from Microsoft’s Information Platform</description>
<lastBuildDate>Tue, 24 Jan 2017 22:08:12 +0000</lastBuildDate>
<language>en-US</language>
<sy:updatePeriod>hourly</sy:updatePeriod>
<sy:updateFrequency>1</sy:updateFrequency>
<item>
<title>SQL Server named DBMS of the Year by DB-Engines</title>
<link>https://blogs.technet.microsoft.com/dataplatforminsider/2017/01/23/sql-server-named-dbms-of-the-year-by-db-engines/</link>
<comments>https://blogs.technet.microsoft.com/dataplatforminsider/2017/01/23/sql-server-named-dbms-of-the-year-by-db-engines/#respond</comments>
<pubDate>Mon, 23 Jan 2017 18:30:00 +0000</pubDate>
<dc:creator><![CDATA[SQL Server Team]]></dc:creator>
<category><![CDATA[Uncategorized]]></category>
<guid isPermaLink="false">https://blogs.technet.microsoft.com/dataplatforminsider/?p=18836</guid>
<description><![CDATA[SQL Server was recently named DBMS of the Year by DB-Engines. DB-Engines is an initiative to collect and present information on database management systems, and provides a widely accepted popularity ranking of database management systems, and has been created and maintained by solid IT, an Austrian company with a special focus on database consulting and... <a aria-label="read more about SQL Server named DBMS of the Year by DB-Engines" href="https://blogs.technet.microsoft.com/dataplatforminsider/2017/01/23/sql-server-named-dbms-of-the-year-by-db-engines/" class="read-more">Read more</a>]]></description>
<content:encoded><![CDATA[<p>SQL Server was recently named DBMS of the Year by DB-Engines. DB-Engines is an initiative to collect and present information on database management systems, and provides a widely accepted popularity ranking of database management systems, and has been created and maintained by solid IT, an Austrian company with a special focus on database consulting and software development. In their award <a href="http://db-engines.com/en/blog_post/67">announcement</a> recognizing SQL Server as the DBMS gaining the most popularity in 2016, DB-Engines cites the “release of SQL Server 2016 and the announcement to port SQL Server to Linux” and also highlights SQL Server’s strength in their ranking categories of job offers, LinkedIn profiles, and other <a href="http://db-engines.com/en/ranking_definition">ranking components</a>.<a href="https://msdnshared.blob.core.windows.net/media/2017/01/SQL-Server-DBMS.png"><img src="https://msdnshared.blob.core.windows.net/media/2017/01/SQL-Server-DBMS.png" alt="SQL-Server-DBMS.png" width="280" height="280" class="size-full wp-image-18825 aligncenter" /></a></p>
<p> </p>
<p>We strive to deliver choice to our customers and to continually empower them to do more. <a href="https://blogs.technet.microsoft.com/dataplatforminsider/2016/06/01/sql-server-2016-is-generally-available-today/">SQL Server 2016</a> enables industry-leading performance and security [1] with built-in business intelligence on any device, as well as in-database advanced analytics, at industry-leading TCO. In 2016, we also <a href="https://blogs.technet.microsoft.com/dataplatforminsider/2016/03/31/microsoft-sql-server-developer-edition-is-now-free/">made our fully-featured SQL Server Developer Edition free</a> for development and testing. [2] With <a href="https://blogs.technet.microsoft.com/dataplatforminsider/2016/11/16/sql-server-2016-service-pack-1-generally-available/">SQL Server 2016 Service Pack 1</a>, we have made key innovations more accessible to developers and organizations across SQL Server editions, making it easier than ever to build advanced applications that scale across editions. Finally, the <a href="https://www.microsoft.com/en-us/sql-server/sql-server-vnext-including-Linux">next release of SQL Server</a>, currently in public preview, brings even more choice to developers, with the abilities to run SQL Server on Linux, Docker Engine, macOS on Docker for Mac, in addition to Windows.</p>
<p>Thank you to DB-Engines for this award, and to our customers for your continued support. We are grateful to have such a passionate user community. Please feel free to share your input about SQL Server <a href="https://connect.microsoft.com/SQLServer">using Microsoft’s Connect tool</a>.</p>
<p>For those who haven’t used SQL Server 2016 or SQL Server v.Next yet, here are a few resources to learn more:</p>
<ul>
<li><a href="https://www.microsoft.com/sql-server/">Keep up with the latest news and information on the SQL Server webpage</a></li>
<li><a href="https://blogs.technet.microsoft.com/dataplatforminsider/2016/11/16/announcing-sql-server-on-linux-public-preview-first-preview-of-next-release-of-sql-server/">Get started today with public preview of next version of SQL Server on Linux or Windows</a></li>
<li><a href="https://www.microsoft.com/en-us/sql-server/developer-tools">Download SQL Server Developer Edition and our developer tools</a> and <a href="https://www.microsoft.com/en-us/sql-server/developer-get-started/">build an app using SQL Server</a></li>
</ul>
<hr align="left" size="1" width="33%" />
<p>[1] National Institute of Standards and Technology Comprehensive Vulnerability Database, 2016.</p>
<p>[2] Development and testing only; not for production or use with production data</p>
]]></content:encoded>
<wfw:commentRss>https://blogs.technet.microsoft.com/dataplatforminsider/2017/01/23/sql-server-named-dbms-of-the-year-by-db-engines/feed/</wfw:commentRss>
<slash:comments>0</slash:comments>
</item>
<item>
<title>December 2016 Leaderboard of Database Systems contributors on MSDN</title>
<link>https://blogs.technet.microsoft.com/dataplatforminsider/2017/01/23/december-2016-leaderboard-of-database-systems-contributors-on-msdn/</link>
<comments>https://blogs.technet.microsoft.com/dataplatforminsider/2017/01/23/december-2016-leaderboard-of-database-systems-contributors-on-msdn/#respond</comments>
<pubDate>Mon, 23 Jan 2017 18:00:00 +0000</pubDate>
<dc:creator><![CDATA[SQL Server Team]]></dc:creator>
<category><![CDATA[Uncategorized]]></category>
<guid isPermaLink="false">https://blogs.technet.microsoft.com/dataplatforminsider/?p=18795</guid>
<description><![CDATA[This post was authored by Rahul Venkatraj, Program Manager, Data Platform Group We continue to receive encouraging comments from the community on the Leaderboard. Thank you. Many congratulations to the top-10 contributors featured on our December leaderboard! Olaf Helper and Alberto Morillo top the Overall and Cloud database this month. Six of this month’s Overall Top-10... <a aria-label="read more about December 2016 Leaderboard of Database Systems contributors on MSDN" href="https://blogs.technet.microsoft.com/dataplatforminsider/2017/01/23/december-2016-leaderboard-of-database-systems-contributors-on-msdn/" class="read-more">Read more</a>]]></description>
<content:encoded><![CDATA[<p>This post was authored by Rahul Venkatraj, Program Manager, Data Platform Group</p>
<p>We continue to receive encouraging comments from the community on the <a href="https://azure.microsoft.com/en-us/blog/sql-leaderboard-msdn-contributors/">Leaderboard</a>. Thank you.<br />
Many congratulations to the top-10 contributors featured on our December leaderboard!</p>
<p><a href="https://msdnshared.blob.core.windows.net/media/2017/01/1981fe82-a692-4d99-af8a-b958b48ad637.png"><img title="1981fe82-a692-4d99-af8a-b958b48ad637" style="padding-top: 0px;padding-left: 0px;padding-right: 0px;border-width: 0px" border="0" alt="1981fe82-a692-4d99-af8a-b958b48ad637" src="https://msdnshared.blob.core.windows.net/media/2017/01/1981fe82-a692-4d99-af8a-b958b48ad637_thumb.png" width="698" height="394" /></a></p>
<p>Olaf Helper and Alberto Morillo top the Overall and Cloud database this month. Six of this month’s Overall Top-10 (including all of the top three) featured in last month’s Overall Top-10 as well, and four others are new entrants.</p>
<p>The following continues to be the points hierarchy (in decreasing order of points):</p>
<p><a href="https://msdnshared.blob.core.windows.net/media/2017/01/525595ff-1d6b-463e-b94d-7d2afd9f6572.png"><img title="525595ff-1d6b-463e-b94d-7d2afd9f6572" style="padding-top: 0px;padding-left: 0px;padding-right: 0px;border-width: 0px" border="0" alt="525595ff-1d6b-463e-b94d-7d2afd9f6572" src="https://msdnshared.blob.core.windows.net/media/2017/01/525595ff-1d6b-463e-b94d-7d2afd9f6572_thumb.png" width="698" height="394" /></a></p>
<p>For questions related to this leaderboard, please write to <a>
[email protected]</a>.</p>
]]></content:encoded>
<wfw:commentRss>https://blogs.technet.microsoft.com/dataplatforminsider/2017/01/23/december-2016-leaderboard-of-database-systems-contributors-on-msdn/feed/</wfw:commentRss>
<slash:comments>0</slash:comments>
</item>
<item>
<title>SQL Server next version CTP 1.2 now available</title>
<link>https://blogs.technet.microsoft.com/dataplatforminsider/2017/01/20/sql-server-next-version-ctp-1-2-now-available/</link>
<comments>https://blogs.technet.microsoft.com/dataplatforminsider/2017/01/20/sql-server-next-version-ctp-1-2-now-available/#respond</comments>
<pubDate>Fri, 20 Jan 2017 18:00:00 +0000</pubDate>
<dc:creator><![CDATA[SQL Server Team]]></dc:creator>
<category><![CDATA[Uncategorized]]></category>
<guid isPermaLink="false">https://blogs.technet.microsoft.com/dataplatforminsider/?p=18806</guid>
<description><![CDATA[As part of our rapid preview model, Microsoft is excited to announce that the next version of SQL Server (SQL Server v.Next) Community Technology Preview (CTP) 1.2 is now available on both Windows and Linux. In CTP 1.2 we implemented bug fixes and added support for SQL Server v.Next on SUSE Linux Enterprise Server. You... <a aria-label="read more about SQL Server next version CTP 1.2 now available" href="https://blogs.technet.microsoft.com/dataplatforminsider/2017/01/20/sql-server-next-version-ctp-1-2-now-available/" class="read-more">Read more</a>]]></description>
<content:encoded><![CDATA[<p>As part of our rapid preview model, Microsoft is excited to announce that the next version of SQL Server (SQL Server v.Next) Community Technology Preview (CTP) 1.2 is now available on both Windows and Linux. In CTP 1.2 we implemented bug fixes and added support for SQL Server v.Next on SUSE Linux Enterprise Server. You can try the preview in your development and test environments now, or apply to join the SQL Server Early Adoption Program to get support for implementing SQL Server v.Next in production.</p>
<h2>Key CTP 1.2 enhancement: Support for SUSE Linux Enterprise</h2>
<p>In SQL Server v.Next, a key design principle has been to provide customers with choice about how to develop and deploy SQL Server applications: using technologies they love like Java, .NET, PHP, Python, R and Node.js, all on the platform of their choosing. Now in CTP 1.2, Microsoft is bringing the power of SQL Server to SUSE Linux Enterprise Server, providing more deployment options and a streamlined acquisition process.</p>
<p>Said Kristin Kinan, Global Alliance Director, Public Cloud at SUSE, “<b>We’re thrilled that Microsoft is announcing support for SQL Server v.Next on SUSE Linux Enterprise Linux. SQL Server and SUSE customers will now be able to run performant, secured SQL Server applications with reliable, cost-effective infrastructure from SUSE.”</b></p>
<p>You can get started with SQL Server on SUSE Linux Enterprise Server v12 SP2 using the <a href="https://docs.microsoft.com/en-us/sql/linux/sql-server-linux-setup-suse-linux-enterprise-server">installation directions</a>. To learn more about how SQL Server runs on SUSE Linux Enterprise Server and container platforms, you can register for <a href="https://www.brighttalk.com/webcast/11477/243417#/utm_source=microsoft&utm_campaign=vnextlaunch&utm_medium=blog">this upcoming webinar</a> that will take place on February 15, 2017. For additional detail on CTP 1.2, please visit <a href="https://msdn.microsoft.com/en-US/library/mt788653(SQL.130).aspx">What’s New in SQL Server v.Next</a>, <a href="https://msdn.microsoft.com/en-US/library/mt788652(SQL.130).aspx">Release Notes</a> and <a href="https://docs.microsoft.com/en-us/sql/linux/">Linux documentation</a>.</p>
<h2>SQL Server Early Adoption Program (EAP)</h2>
<p>Today we also announced the SQL Server v.Next Early Adoption Program (EAP). The Early Adoption Program is designed to help customers and partners evaluate new features in SQL Server v.Next, and to build and deploy applications for SQL Server v.Next on Windows and Linux. Qualified applicants will receive technical assistance from Microsoft engineers to deploy and support an application in production before general availability, or to build or modernize an application for SQL Server v.Next. Read the <a href="https://blogs.technet.microsoft.com/dataplatforminsider/2017/01/20/announcing-the-sql-server-v-next-early-adoption-program">detailed blog on EAP</a> to learn more about all the benefits of this program and how to get started.</p>
<h2>Get SQL Server v.Next CTP 1.2 today!</h2>
<p>Try the preview of the next release of SQL Server today! Get started with the preview of SQL Server with <a href="http://www.microsoft.com/sql-server/developer-get-started">our developer tutorials</a> that show you how to install and use SQL Server v.Next on macOS, Docker, Windows, and Linux and quickly build an app in a programming language of your choice.</p>
<ul>
<li><a href="https://aka.ms/eapsignup">Sign up for the Early Adoption Program (EAP)</a></li>
<li><a href="https://docs.microsoft.com/en-us/sql/linux/sql-server-linux-setup-suse-linux-enterprise-server">Install on SUSE Linux Enterprise Server</a></li>
<li><a href="https://docs.microsoft.com/en-us/sql/linux/sql-server-linux-setup-red-hat">Install on Red Hat Enterprise Linux</a></li>
<li><a href="https://docs.microsoft.com/en-us/sql/linux/sql-server-linux-setup-ubuntu">Install on Ubuntu Linux</a></li>
<li><a href="https://docs.microsoft.com/en-us/sql/linux/">Pull and run a Docker container on Linux, Windows, or macOS</a></li>
<li><a href="https://www.microsoft.com/evalcenter/evaluate-sql-server-vnext-ctp">Download the preview for Windows</a></li>
<li><a href="https://azure.microsoft.com/en-us/marketplace/?term=%22SQL+Server+vNext%22">Create a SQL Server on Linux virtual machine in Azure</a></li>
<li><a href="https://www.brighttalk.com/webcast/11477/243417#/utm_source=microsoft&utm_campaign=vnextlaunch&utm_medium=blog">Register for the Introduction to SQL Server on SUSE Linux Enterprise Server Webinar</a></li>
</ul>
<p>Have questions? Join the discussion of SQL Server v.Next at <a href="https://social.msdn.microsoft.com/Forums/sqlserver/en-US/home">MSDN</a>. If you run into an issue or would like to make a suggestion, you can let us know through <a href="https://connect.microsoft.com/SQLServer">Connect</a>. We look forward to hearing from you!</p>
]]></content:encoded>
<wfw:commentRss>https://blogs.technet.microsoft.com/dataplatforminsider/2017/01/20/sql-server-next-version-ctp-1-2-now-available/feed/</wfw:commentRss>
<slash:comments>0</slash:comments>
</item>
<item>
<title>Announcing the SQL Server v.Next Early Adoption Program</title>
<link>https://blogs.technet.microsoft.com/dataplatforminsider/2017/01/20/announcing-the-sql-server-v-next-early-adoption-program/</link>
<comments>https://blogs.technet.microsoft.com/dataplatforminsider/2017/01/20/announcing-the-sql-server-v-next-early-adoption-program/#respond</comments>
<pubDate>Fri, 20 Jan 2017 17:30:00 +0000</pubDate>
<dc:creator><![CDATA[SQL Server Team]]></dc:creator>
<category><![CDATA[Uncategorized]]></category>
<guid isPermaLink="false">https://blogs.technet.microsoft.com/dataplatforminsider/?p=18745</guid>
<description><![CDATA[What is the SQL Server Early Adoption Program (SQL EAP)? The SQL Server Early Adoption Program (SQL EAP) is a Microsoft program started in January 2017 to help both customers and partners adopt the next version of SQL Server before general availability. Who is SQL EAP for? If you are interested in adopting SQL Server... <a aria-label="read more about Announcing the SQL Server v.Next Early Adoption Program" href="https://blogs.technet.microsoft.com/dataplatforminsider/2017/01/20/announcing-the-sql-server-v-next-early-adoption-program/" class="read-more">Read more</a>]]></description>
<content:encoded><![CDATA[<h2>What is the SQL Server Early Adoption Program (SQL EAP)?</h2>
<p>The <a href="https://aka.ms/eapsignup">SQL Server Early Adoption P</a>rogram (SQL EAP) is a Microsoft program started in January 2017 to help both <b>customers and partners</b> adopt the next version of SQL Server before general availability.</p>
<h2>Who is SQL EAP for?</h2>
<p>If you are interested in<b> adopting SQL Server v.Next on Windows or Linux in production,</b> then SQL EAP is for you.<b> </b>SQL EAP is also for partners who want to build SI (system integrator) offerings and ISV (independent software vendor) applications using SQL Server v.Next. Upon successful validation, these applications and solutions can be supported in production prior to the general availability release. Customers and partners who would like to <b>validate new features </b>such as Adaptive Query Processing and High Availability (HA) on Linux<b> </b>are an especially good fit for SQL EAP. Enroll for the program <a href="https://aka.ms/eapsignup">here</a>.</p>
<h2>What are the benefits?</h2>
<ul>
<li>Through the program you will have direct access to the engineering team through a <b>Program Manager Buddy.</b> Your PM Buddy is there as a primary contact within the development team to help connect you to the right people to help your solution adopt SQL Server v.Next. Typically, PM Buddies communicate with the customer via email and regularly scheduled meetings. PM Buddies help scope the project when the customer first joins SQL EAP so that there is common understanding of the schedule and requirements.</li>
<li>SQL EAP participants have the opportunity to bring your workload to the <b>SQL Customer Advisory Team Customer Lab </b>to directly engage and test with the SQL Server team.</li>
<li>Customers in SQL EAP will be able to try out new features, sometimes before the public gets to see them, and provide the feedback directly to the engineering team. They will have the opportunity to <b>provide input into the prioritization of product requirements</b> for SQL Server v.Next via regular surveys. Participants will also be able to discuss feature design with PMs.</li>
<li>You will have access to a <b>private Yammer group</b> for SQL EAP customers to communicate with one another and the engineering team, helping customers learn from each other. Content on the private Yammer group will be considered confidential and covered by the Microsoft NDA required to participate in SQL EAP.</li>
<li>Customers going into production will be fully supported by Microsoft Support before general availability. A <b>special support channel</b> is provided to raise cases for SQL Server v.Next. The SQL Server engineering team will be backing up the Microsoft Support team to provide assistance as needed. Customers in production will also have support for release-to-release upgrades.</li>
</ul>
<h2>What are the requirements?</h2>
<ul>
<li>Complete the form <a href="https://aka.ms/eapsignup">here</a>. Enrollments will be evaluated and a PM Buddy aligned to access workload validation and those on track for production deployments.</li>
<li>An NDA with Microsoft will be required to participate in the program. If you do not already have an NDA, we will help to get one signed.</li>
<li>Customers in SQL EAP that are going in production will need to sign a EULA amendment that grants them the permission to use the software in production.</li>
</ul>
<h2>Get Started Today:</h2>
<p>Apply for the program <a href="https://aka.ms/eapsignup">here</a>!</p>
<h2>Join the Webinar:</h2>
<p>We are having a webinar called SQL Server on Linux Next Steps on 1/24 at 6pm PST and also 1/31 at 9pm PST. Please join us to learn more about SQL Early Adoption Program. The webinar covers:</p>
<ul>
<li>The latest updates to SQL Server v.Next</li>
<li>How SQL Server 2016 v.Next can improve your applications and solutions</li>
<li>Our SQL Early Adoption Program where you can get advice from our technical experts and get technical resources to help upgrade or migrate your applications to SQL Server v.Next.</li>
</ul>
<p>We will have a Q&A session at the end of the webinar.</p>
<p><a href="https://join-noam.broadcast.skype.com/microsoft.com/46a681fefce04a408fcfce34aca08f27">Link to Skype Broadcast 1/24 at 6pm PST</a><br />
<a href="https://join-noam.broadcast.skype.com/microsoft.com/fca1fbb867024eb5abf1bfd9237e3161">Link to Skype Broadcast 1/31 at 9pm PST</a></p>
<h2>Learn More:</h2>
<ul>
<li><a href="https://www.microsoft.com/en-us/sql-server/sql-server-vnext-including-Linux">Download the preview</a></li>
<li><a href="https://docs.microsoft.com/en-us/sql/linux/sql-server-linux-performance-get-started">Read the documentation</a></li>
<li><a href="https://www.microsoft.com/en-us/sql-server/developer-get-started/">Get started with code samples</a></li>
<li><a href="https://twitter.com/SQLServer">Follow us on Twitter at @SQLServer</a></li>
<li><a href="https://helsinki.catalysis.com/registration.aspx">Sign up to stay informed about SQL Server on Linux</a></li>
</ul>
]]></content:encoded>
<wfw:commentRss>https://blogs.technet.microsoft.com/dataplatforminsider/2017/01/20/announcing-the-sql-server-v-next-early-adoption-program/feed/</wfw:commentRss>
<slash:comments>0</slash:comments>
</item>
<item>
<title>Announcing Columnstore Indexes and Query Store support in Database Engine Tuning Advisor</title>
<link>https://blogs.technet.microsoft.com/dataplatforminsider/2017/01/10/announcing-columnstore-indexes-and-query-store-support-in-database-engine-tuning-advisor/</link>
<comments>https://blogs.technet.microsoft.com/dataplatforminsider/2017/01/10/announcing-columnstore-indexes-and-query-store-support-in-database-engine-tuning-advisor/#comments</comments>
<pubDate>Tue, 10 Jan 2017 20:00:32 +0000</pubDate>
<dc:creator><![CDATA[SQL Server Team]]></dc:creator>
<category><![CDATA[Uncategorized]]></category>
<guid isPermaLink="false">https://blogs.technet.microsoft.com/dataplatforminsider/?p=18085</guid>
<description><![CDATA[The latest version of Microsoft SQL Server Database Engine Tuning Advisor (DTA) supports two new features: (a) Ability to recommend columnstore indexes in addition to rowstore indexes, and (b) Ability to automatically select a workload to tune from the Query Store. These new features are available when tuning for SQL Server 2016 Database Engine (or later)... <a aria-label="read more about Announcing Columnstore Indexes and Query Store support in Database Engine Tuning Advisor" href="https://blogs.technet.microsoft.com/dataplatforminsider/2017/01/10/announcing-columnstore-indexes-and-query-store-support-in-database-engine-tuning-advisor/" class="read-more">Read more</a>]]></description>
<content:encoded><![CDATA[<p>The latest version of Microsoft SQL Server <a href="https://msdn.microsoft.com/library/hh231122.aspx">Database Engine Tuning Advisor</a> (<strong>DTA</strong>) supports two new features: <em>(a)</em> Ability to <a href="https://msdn.microsoft.com/library/mt805316.aspx">recommend columnstore indexes</a> in addition to rowstore indexes, and <em>(b)</em> Ability to automatically <a href="https://msdn.microsoft.com/library/mt805317.aspx">select a workload to tune from the Query Store</a>. These new features are available when tuning for SQL Server 2016 Database Engine (or later) versions.</p>
<h4>Recommendations for columnstore indexes</h4>
<p>Data warehousing and analytic workloads often need to scan large amounts of data, and can greatly benefit from <a href="https://msdn.microsoft.com/en-us/library/gg492088.aspx">columnstore indexes</a>. In contrast, rowstore<span> (B+-tree) indexes are most effective for queries that access relatively small amounts of data searching for a particular range of values. Since rowstore indexes can deliver rows in sorted order, they can also reduce the cost of sorting in query execution plans</span>. Therefore, the choice of which rowstore and columnstore indexes to build for your database is dependent on your application’s workload.</p>
<p>The latest version of DTA can analyze the workload and recommend a suitable <strong>combination of rowstore and columnstore indexes</strong> to build on the tables referenced by the workload. <a href="https://msdn.microsoft.com/library/mt805801.aspx">This article</a> highlights the performance improvements achieved on real customer workloads by using DTA to recommend a combination of rowstore and columnstore indexes.</p>
<h4>Tune Database using Workload from SQL Server Query Store</h4>
<p>The <a href="https://msdn.microsoft.com/en-US/library/dn817826.aspx">Query Store</a> feature in SQL Server automatically captures a history of queries, plans, and runtime statistics, and persists this information along with a database. It stores query execution statistics summarized over time intervals so you can see database usage patterns and understand when query plan changes happened on the server. DTA now supports a new option to analyze the Query Store to automatically select an appropriate workload for tuning. For many DTA users, this can take away the burden of having to collect a suitable workload file using SQL Server Profiler. This feature is only available if the database has Query Store turned on.</p>
<h4>Next Steps</h4>
<p><a href="https://msdn.microsoft.com/library/mt238290.aspx">Download the latest version of Database Engine Tuning Advisor</a><br />
For additional documentation on these features see also:<br />
<a href="https://msdn.microsoft.com/library/mt805316.aspx">Columnstore Index Recommendations in Database Engine Tuning Advisor (DTA)</a><br />
<a href="https://msdn.microsoft.com/library/mt805317.aspx">Tuning Database Using Workload from Query Store</a><br />
<a href="https://msdn.microsoft.com/library/mt805801.aspx">Performance Improvements using Database Engine Tuning Advisor (DTA) recommendations</a></p>
]]></content:encoded>
<wfw:commentRss>https://blogs.technet.microsoft.com/dataplatforminsider/2017/01/10/announcing-columnstore-indexes-and-query-store-support-in-database-engine-tuning-advisor/feed/</wfw:commentRss>
<slash:comments>3</slash:comments>
</item>
<item>
<title>How six lines of code + SQL Server can bring Deep Learning to ANY App</title>
<link>https://blogs.technet.microsoft.com/dataplatforminsider/2017/01/05/how-six-lines-of-code-sql-server-can-bring-deep-learning-to-any-app/</link>
<comments>https://blogs.technet.microsoft.com/dataplatforminsider/2017/01/05/how-six-lines-of-code-sql-server-can-bring-deep-learning-to-any-app/#comments</comments>
<pubDate>Thu, 05 Jan 2017 17:00:07 +0000</pubDate>
<dc:creator><![CDATA[Rimma Nehme]]></dc:creator>
<category><![CDATA[Uncategorized]]></category>
<guid isPermaLink="false">https://blogs.technet.microsoft.com/dataplatforminsider/?p=18715</guid>
<description><![CDATA[The link to the source code is here. The What Part Deep Learning is a hot buzzword of today. The recent results and applications are incredibly promising, spanning areas such as speech recognition, language understanding and computer vision. Indeed, Deep Learning is now changing the very customer experience around many of Microsoft’s products, including HoloLens,... <a aria-label="read more about How six lines of code + SQL Server can bring Deep Learning to ANY App" href="https://blogs.technet.microsoft.com/dataplatforminsider/2017/01/05/how-six-lines-of-code-sql-server-can-bring-deep-learning-to-any-app/" class="read-more">Read more</a>]]></description>
<content:encoded><![CDATA[<p><a href="https://msdnshared.blob.core.windows.net/media/2017/01/Deep-SQL-Server.png"><img width="640" height="359" title="Deep SQL Server" class="aligncenter" style="float: none;padding-top: 0px;padding-left: 0px;margin-left: auto;padding-right: 0px;margin-right: auto;border-width: 0px" alt="Deep SQL Server" src="https://msdnshared.blob.core.windows.net/media/2017/01/Deep-SQL-Server_thumb.png" border="0" /></a></p>
<p><em>The link to the source code is <a href="https://github.com/Microsoft/SQL-Server-R-Services-Samples/tree/master/Galaxies">here</a>.</em></p>
<h2>The What Part</h2>
<p>Deep Learning is a hot buzzword of today. The recent results and applications are incredibly promising, spanning areas such as speech recognition, language understanding and computer vision. Indeed, Deep Learning is now changing the very customer experience around many of Microsoft’s products, including HoloLens, Skype, Cortana, Office 365, Bing and more. Deep Learning is also a core part of Microsoft’s development platform offerings with an extensive toolset that includes: the Microsoft <a href="http://cntk.ai/">Cognitive Toolkit</a>, the <a href="https://www.microsoft.com/en-us/cloud-platform/cortana-intelligence-suite">Cortana Intelligence Suite</a>, <a href="https://www.microsoft.com/cognitive-services">Microsoft Cognitive Services APIs</a>, <a href="https://azure.microsoft.com/en-us/services/machine-learning/">Azure Machine Learning</a>, the <a href="https://dev.botframework.com/">Bot Framework</a>, and the <a href="https://azure.microsoft.com/en-us/services/bot-service/">Azure Bot Service</a>. Our Deep Learning based <a href="http://www.popsci.com/how-microsofts-machine-learning-breaking-language-barrier">language translation in Skype</a> was recently named one of the <a href="http://www.popsci.com/6-greatest-software-innovations-year">7 greatest software innovations of the year</a> by Popular Science, and this technology has now helped machines achieve <a href="http://blogs.microsoft.com/next/2016/09/13/microsoft-researchers-achieve-speech-recognition-milestone/#sm.0001ozddel7hlfa411f7lw7b80vuz">human-level parity in conversational speech recognition</a>. To learn more about our Deep Learning journey, I encourage you to read a recent blog <a href="https://azure.microsoft.com/en-us/blog/from-a-pc-on-every-desktop-to-deep-learning-in-every-software/">From “A PC on every desktop” to “Deep Learning in every software”.</a></p>
<p>The applications of Deep Learning technology are truly so far reaching that the new mantra, of <em>Deep Learning in Every Software</em>, may well become a reality within this decade. The venerable SQL Server DBMS is no exception. <strong>Can SQL Server do Deep Learning?</strong> The response to this is enthusiastic “<strong>yes</strong>!” With the <a href="https://blogs.technet.microsoft.com/dataplatforminsider/2016/11/16/announcing-the-next-generation-of-databases-and-data-lakes-from-microsoft/">public preview of the next release of SQL Server</a>, we’ve added significant improvements into <a href="https://msdn.microsoft.com/en-us/library/mt604885.aspx">R Services inside SQL Server</a> including a very powerful set of machine learning functions that are used by our own product teams across Microsoft. <strong>This brings new machine learning and deep neural network functionality with increased speed, performance and scale to database applications built on SQL Server</strong>. We have just recently <a href="https://blogs.technet.microsoft.com/dataplatforminsider/2016/10/11/1000000-predictions-per-second/">showcased SQL Server running more than one million R predictions per second</a>, using <a href="https://blogs.technet.microsoft.com/dataplatforminsider/2016/10/17/sql-server-as-a-machine-learning-model-management-system/">SQL Server as a Machine Learning Model Management System</a> and I encourage you all to try out R examples and machine learning templates for <a href="https://github.com/Microsoft/SQL-Server-R-Services-Samples">SQL Server on GitHub</a>.</p>
<p>In this blog, I wanted to address the finer points of the matter – the what, the why and the how part of Deep Learning in SQL Server. With this new clarity, it will be easier to see a picture of the road forward for data-driven machine intelligence using such a powerful data platform like SQL Server.</p>
<h2>The Why Part</h2>
<p>Today, every company is a data company, and every app is a data app.</p>
<p>When you put intelligence (AI, ML, AA, etc.) close to where the data lives, then every app becomes an intelligent app. SQL Server can help developers and customers everywhere realize the holy grail of deep learning in their applications with just a few lines of code. It enables It enables data developers to deploy mission critical operational systems that embed deep learning models. So here are the 10 whys for deep learning in SQL Server.</p>
<p>The 10 Whys of Deep Learning inside SQL Server</p>
<ol>
<li>By pushing intelligence close to where your data lives (i.e., SQL Server), you get security, compliance, privacy, encryption, <a href="https://msdn.microsoft.com/en-us/library/ff487003.aspx">master data services</a>, availability groups, advanced BI, in-memory, virtualization, geo-spatial, temporal, graph capabilities and other world-class features.</li>
<li>You can do both near “real-time intelligence” or “batch intelligence” (similar in spirit to OLTP and OLAP, but applied to Deep Learning and intelligence).</li>
<li>Your apps built on top of SQL Server don’t need to change to take advantage of Deep Learning, and a multitude of apps (web, mobile, IoT) can share the same deep learning models without duplicating code.</li>
<li>You can exploit a number of functionalities that come in machine learning libraries (e.g., MicrosoftML) that will drive the productivity of your data scientists, developers and DBAs and business overall. This might be faster and far more efficient than doing it in the house.</li>
<li>You can develop predictable solutions that can evolve/scale up as you need. With the latest service pack of SQL Server, many features that were only available in the Enterprise Edition are now available in the Standard/Express/Web Edition of SQL Server. That means you can do Deep Learning using a standard SQL Server without high costs.</li>
<li>You can use heterogeneous external data sources (via Polybase) for training and inference of deep models.</li>
<li>You can create versatile data simulations and what-if scenarios inside SQL Server and then train a variety of rich Deep Learning models in those simulated worlds to enable intelligence even with a limited training data.</li>
<li>You can operationalize Deep Learning models in a very easy and fast way using stored procedures and triggers.</li>
<li>You get all the tools, monitoring, debugging and ecosystem around SQL Server applicable to intelligence. SQL Server can literally become your <a href="https://blogs.technet.microsoft.com/dataplatforminsider/2016/10/17/sql-server-as-a-machine-learning-model-management-system/">Machine Learning Management System</a> and handle the entire life cycle of DNN models along with data.</li>
<li>You can generate new knowledge and insights on the data you are storing already and anyways without having any impact on your transactional workload (via <a href="http://blogs.microsoft.com/blog/2016/03/10/sql-server-2016-the-database-for-mission-critical-intelligence/#sm.0001ozddel7hlfa411f7lw7b80vuz">HTAP pattern</a>).</li>
</ol>
<p>Let’s be honest, nobody buys a DBMS for the sake of DBMS. People buy it for what it enables you to do. By putting deep learning capabilities inside SQL Server, we can scale artificial intelligence and machine learning both in traditional sense (scale of data, throughput, latency), but we also scale it in terms of productivity (low barrier to adoption and lower learning curve). The value that it brings results in so many shapes and forms – time, better experience, productivity, lower $ cost, higher $ revenue, opportunity, higher business aspirations, thought-leadership in an industry, etc.</p>
<p>Real-life applications of Deep Learning running inside SQL Server span banking, healthcare, finance, manufacturing, retail, e-commerce and IoT. With applications like fraud detection, disease prediction, power consumption prediction, personal analytics, you have the ability to transform existing industries and apps. That also means whatever workloads you are running using SQL Server, be it CRM, ERP, DW, OLTP, BD… you can add Deep Learning to them almost seamlessly. Furthermore, it’s not just about doing deep learning standalone, but it’s rather about combining it with all kinds of data and analytics that SQL Server is so great at (e.g., processing structured data, JSON data, geo-spatial data, graph data, external data, temporal data). All that is really left to be added to this mix is… your creativity.</p>
<p><a href="https://msdnshared.blob.core.windows.net/media/2017/01/SQL-Server-and-Deep-Learning.png"><img width="640" height="359" title="SQL Server and Deep Learning" class="aligncenter" style="float: none;padding-top: 0px;padding-left: 0px;margin-left: auto;padding-right: 0px;margin-right: auto;border-width: 0px" alt="SQL Server and Deep Learning" src="https://msdnshared.blob.core.windows.net/media/2017/01/SQL-Server-and-Deep-Learning_thumb.png" border="0" /></a></p>
<h2>The How Part</h2>
<p>Here is a great scenario to show all of this in reality. I am going to use an example of predicting galaxy classes from image data – using the power of <strong>Microsoft R</strong> and its new <a href="https://msdn.microsoft.com/en-us/microsoft-r/microsoftml-introduction">MicrosoftML</a> package for machine learning (which has been built by our Algorithms and Data Science team). And I am going to do all this in <strong>SQL Server</strong> with <a href="https://msdn.microsoft.com/en-us/library/mt604845.aspx">R Services</a> on a readily available <a href="https://azure.microsoft.com/en-us/blog/azure-n-series-general-availability-on-december-1/">Azure NC VM</a>. I am going to classify the images of galaxies and other celestial objects into 13 different classes based on the taxonomy created by astronomers – mainly elliptical and spirals and then various sub-categories within them. The shape and other visual features of galaxies change as they evolve. Studying the shapes of galaxies and classifying them appropriately helps scientists learn how the universe is evolving. It is very easy for us humans to look at these images and put them in the right buckets based on the visual features. But in order to scale it to the 2 trillion known galaxies I need help from machine learning and techniques like deep neural networks – so that is exactly what I am going to use. It’s not a big leap to imagine that instead of astronomy data, we have healthcare data or financial data or IoT data and we are trying to make predictions on that data.</p>
<h3>An app</h3>
<p>Imagine a simple web app that loads images from a folder and then classifies them into different categories – spiral or elliptical and then sub-types with those categories (e.g., is it a regular spiral or does it have a handlebar structure in the center).</p>
<p><a href="https://msdnshared.blob.core.windows.net/media/2017/01/Use-Deep-Neural-Nets.png"><img width="446" height="99" title="Use Deep Neural Nets" class="aligncenter" style="float: none;padding-top: 0px;padding-left: 0px;margin-left: auto;padding-right: 0px;margin-right: auto;border-width: 0px" alt="Use Deep Neural Nets" src="https://msdnshared.blob.core.windows.net/media/2017/01/Use-Deep-Neural-Nets_thumb.png" border="0" /></a></p>
<p>The classification can be done incredibly fast on vast amounts of images. Here is an example output:</p>
<p><a href="https://msdnshared.blob.core.windows.net/media/2017/01/Classifier.png"><img width="640" height="467" title="Classifier" class="aligncenter" style="float: none;padding-top: 0px;padding-left: 0px;margin-left: auto;padding-right: 0px;margin-right: auto;border-width: 0px" alt="Classifier" src="https://msdnshared.blob.core.windows.net/media/2017/01/Classifier_thumb.png" border="0" /></a><br />
The first two columns are the elliptical types and the others are of different spiral types.</p>
<h3>So how does this simple app do this amazingly complex computation?</h3>
<p>The code for such an app actually isn’t doing much – it just writes the paths to the new files to classify into a database table (the rest of the app code is plumbing and page layout, etc).</p>
<pre class="prettyprint">SqlCommand Cmd = new SqlCommand("INSERT INTO [dbo].[GalaxiesToScore] ([path] ,[PredictedLabel]) "</pre>
<p> </p>
<pre class="prettyprint"><a href="https://msdnshared.blob.core.windows.net/media/2017/01/CodeDemo.png"><img width="1024" height="193" title="CodeDemo" class="aligncenter" style="float: none;padding-top: 0px;padding-left: 0px;margin-left: auto;padding-right: 0px;margin-right: auto;border-width: 0px" alt="CodeDemo" src="https://msdnshared.blob.core.windows.net/media/2017/01/CodeDemo_thumb.png" border="0" /></a></pre>
<h3>What is happening in the database?</h3>
<h4>Prediction and operationalization part:</h4>
<p>Let’s look at the table where the app writes the image paths. It contains a column with paths to the galaxy images, and a column to store the predicted classes of galaxies. As soon as a new row of data gets entered into this table, a trigger gets executed.</p>
<p><a href="https://msdnshared.blob.core.windows.net/media/2017/01/Image-Path-Table-1.png"><img width="1024" height="356" title="Image Path Table 1" class="aligncenter" style="float: none;padding-top: 0px;padding-left: 0px;margin-left: auto;padding-right: 0px;margin-right: auto;border-width: 0px" alt="Image Path Table 1" src="https://msdnshared.blob.core.windows.net/media/2017/01/Image-Path-Table-1_thumb.png" border="0" /></a></p>
<p> </p>
<p>The trigger in turn invokes a stored procedure – <em>PredictGalaxiesNN</em> as shown below (with R script portion embedded inside the stored proc):</p>
<p><a href="https://msdnshared.blob.core.windows.net/media/2017/01/Image-Path-Table-2.png"><img width="1024" height="461" title="Image Path Table 2" class="aligncenter" style="float: none;padding-top: 0px;padding-left: 0px;margin-left: auto;padding-right: 0px;margin-right: auto;border-width: 0px" alt="Image Path Table 2" src="https://msdnshared.blob.core.windows.net/media/2017/01/Image-Path-Table-2_thumb.png" border="0" /></a></p>
<p>This is where the <strong>magic</strong> happens – in these few lines of R code. This R script takes two inputs – the new rows of data (that have not been scored yet) and the model that is stored in a table as varbinary(max). I will talk about how the model got there in a minute. Inside the script, the model gets de-serialized and is used by the familiar scoring function (<em>rxPredict</em>) in this line:</p>
<pre class="prettyprint">scores <- rxPredict(modelObject = model_un, data = InputDataSet, extraVarsToWrite="path")</pre>
<p>to score the new rows and then write the scored output out. This is a new variant of <em>rxPredict</em> which understands the ML algorithms included in the new Microsoft ML package. This line</p>
<pre class="prettyprint"> [ library("MicrosoftML") ]</pre>
<p>loads the new package that contains the new ML algorithms. In addition to DNN (the focus of this blog), there are five other powerful ML algorithms in this package – fast linear learner, fast tree, fast forest, one class SVM for anomaly detection, regularized logistic regression (with L1 and L2) and neural nets. <strong>So, with just 6-7 lines of R code, you can enable any app to get the intelligence from a DNN based model</strong>. All that the apps needs to do is connect to SQL Server. By the way, you can now very easily generate a stored procedure for R Code using the <a href="https://msdn.microsoft.com/en-us/library/mt790483.aspx">sqlrutils</a> package.</p>
<h4>What about training the model?</h4>
<p>Where was the model trained? Well, the model was trained in SQL Server as well. However,<em> it does not have to be trained on SQL Server</em> – it could have been trained on a separate machine with a standalone R Server running on it, on-prem or on the cloud. Today we have these new ML algorithms on Windows version of <strong>R Server</strong>, and the support for other platforms is coming soon. I just chose to do the training in the SQL server box here, but I could have done it outside as well. Let’s look at the stored proc with the training code.</p>
<h5>Training code:</h5>
<p>The model training is done in these lines of code.</p>
<p><a href="https://msdnshared.blob.core.windows.net/media/2017/01/Training-Code.png"><img width="640" height="337" title="Training Code" class="aligncenter" style="float: none;padding-top: 0px;padding-left: 0px;margin-left: auto;padding-right: 0px;margin-right: auto;border-width: 0px" alt="Training Code" src="https://msdnshared.blob.core.windows.net/media/2017/01/Training-Code_thumb.png" border="0" /></a></p>
<p>This new function – <strong>rxNeuralNet</strong> from the <strong>MicrosoftML</strong> package for training a DNN. The code looks similar to other R and rx functions – there is a formula, an input dataset, and some other parameters. One of the parameters here is this line “netDefinition = netDefinition”. This is where the neural network is being defined.</p>
<h5>Network definition:</h5>
<p>Here is the DNN definition in this portion of the code:</p>
<p><a href="https://msdnshared.blob.core.windows.net/media/2017/01/DNN-Definition.png"><img width="640" height="387" title="DNN Definition" class="aligncenter" style="float: none;padding-top: 0px;padding-left: 0px;margin-left: auto;padding-right: 0px;margin-right: auto;border-width: 0px" alt="DNN Definition" src="https://msdnshared.blob.core.windows.net/media/2017/01/DNN-Definition_thumb.png" border="0" /></a></p>
<p>Here, a deep neural net is defined using Net# specification language that was created for this purpose. It has 1 input, 1 output and 8 hidden layers. It starts with an input layer of 50×50 pixels and 3 colors (RGB) image data. First hidden layer is a convolution layer where we specify the kernel (small sub-part of the image) size and how many times we want the kernel to map to other kernels (convolute). There are some other layers for more convolutions, and for normalization and pooling that help stabilize the network. And finally, the output layer that maps it to one of the 13 classes. In about 50 lines of Net# specification, I have defined a complex neural network. <a href="https://msdn.microsoft.com/en-us/library/azure/dn906030.aspx">Net# is documented on MSDN</a>.</p>
<h5>Training data size/GPU:</h5>
<p>Here is the R code to do the training.</p>
<p><a href="https://msdnshared.blob.core.windows.net/media/2017/01/R-code.png"><img width="532" height="480" title="R code" class="aligncenter" style="float: none;padding-top: 0px;padding-left: 0px;margin-left: auto;padding-right: 0px;margin-right: auto;border-width: 0px" alt="R code" src="https://msdnshared.blob.core.windows.net/media/2017/01/R-code_thumb.png" border="0" /></a></p>
<p>Some other lines to note here are – ‘training_rows = 238000’. This model was trained on 238K images that we got from <a href="https://blogs.technet.microsoft.com/dataplatforminsider/2016/03/10/mapping-the-universe-with-sql-server/">Sloan Digital Sky Survey dataset</a>. We then created two variants of each image with 45% and 90% rotations. In all there was about 700K images to train on. That’s a lot of image data to train on – so, how long did it take to train it? Well, we were able to train this model in under 4 hours. This is a decent sized machine – 6 cores and 56GB or RAM, but then it also has a powerful Nvidia <strong>Tesla K80 GPU</strong>. In fact, it is an <strong>Azure VM</strong> – the new <strong>NC series GPU VM</strong>, readily available to anyone with an Azure subscription. We were able to leverage the GPU computation by specifying one simple parameter: <em>acceleration = “gpu”.</em> Without GPU, the training takes roughly 10X more time.</p>
<h2>The What Are You Waiting For Part</h2>
<p>So with just a few lines of R code using algorithms from the MicrosoftML package, I was able to train a DNN on tons of image data and operationalize the trained model in SQL using R services such that any app connected to SQL can get this type of Intelligence easily. That’s the power of Microsoft R and the Microsoft ML package in it combined with SQL Server. This is just the beginning, and we are working on adding more algorithms on our quest to democratize the power of AI and machine learning. You can download the <strong>MicrosoftML: Algorithm Cheat Sheet </strong><a href="https://msdn.microsoft.com/en-us/microsoft-r/microsoftml-algorithm-cheat-sheet">here</a> to help you choose the right machine learning algorithm for a predictive analytics model.</p>
<p><a href="https://msdnshared.blob.core.windows.net/media/2017/01/MicrosoftML-Alogorithm-Cheat-Sheet.png"><img width="1024" height="663" title="MicrosoftML Alogorithm Cheat Sheet" class="aligncenter" style="float: none;padding-top: 0px;padding-left: 0px;margin-left: auto;padding-right: 0px;margin-right: auto;border-width: 0px" alt="MicrosoftML Alogorithm Cheat Sheet" src="https://msdnshared.blob.core.windows.net/media/2017/01/MicrosoftML-Alogorithm-Cheat-Sheet_thumb.png" border="0" /></a></p>
<p>Don’t wait, go ahead and give it a try.</p>
<p><a href="https://twitter.com/rimmanehme">@rimmanehme</a></p>
]]></content:encoded>
<wfw:commentRss>https://blogs.technet.microsoft.com/dataplatforminsider/2017/01/05/how-six-lines-of-code-sql-server-can-bring-deep-learning-to-any-app/feed/</wfw:commentRss>
<slash:comments>16</slash:comments>
</item>
<item>
<title>Welcome 2017! Recap of top SQL Server posts of 2016</title>
<link>https://blogs.technet.microsoft.com/dataplatforminsider/2017/01/04/welcome-2017-recap-of-top-sql-server-posts-of-2016/</link>
<pubDate>Wed, 04 Jan 2017 17:00:23 +0000</pubDate>
<dc:creator><![CDATA[SQL Server Team]]></dc:creator>
<category><![CDATA[Uncategorized]]></category>
<guid isPermaLink="false">https://blogs.technet.microsoft.com/dataplatforminsider/?p=18485</guid>
<description><![CDATA[We start off the new year with a quick recap of our most popular posts on this blog channel from the year just concluded. Here are the posts that got the most number of views from our readers during 2016, in increasing order of readership: 10. SQL Server 2016 Express Edition in Windows containers In... <a aria-label="read more about Welcome 2017! Recap of top SQL Server posts of 2016" href="https://blogs.technet.microsoft.com/dataplatforminsider/2017/01/04/welcome-2017-recap-of-top-sql-server-posts-of-2016/" class="read-more">Read more</a>]]></description>
<content:encoded><![CDATA[<p>We start off the new year with a quick recap of our most popular posts on this blog channel from the year just concluded.</p>
<p>Here are the posts that got the most number of views from our readers during 2016, in increasing order of readership:</p>
<h3>10. <a href="https://blogs.technet.microsoft.com/dataplatforminsider/2016/10/13/sql-server-2016-express-edition-in-windows-containers/">SQL Server 2016 Express Edition in Windows containers</a></h3>
<p>In October, we announced the public availability of SQL Server 2016 Express Edition in Windows Containers, a capability that comes in especially handy when you need to quickly create SQL Server instances for development or testing; or to maximize density in a test or production environment (especially in microservice architectures); or when you need to isolate and control apps in a multi-tenant infrastructure.</p>
<h3>9. <a href="https://blogs.technet.microsoft.com/dataplatforminsider/2016/11/16/sql-server-2016-service-pack-1-generally-available/">SQL Server 2016 Service Pack 1 generally available</a></h3>
<p>A couple of months back, we made several new capabilities available as part of the SP1 release of SQL Server 2016. Developers can now build to a common programming surface across all editions – and do tons more, see the graphic below or read the full article for more.</p>
<p><a href="https://msdnshared.blob.core.windows.net/media/2017/01/SQL-Server-2016-Service-Pack-1.png"><img title="SQL Server 2016 Service Pack 1" style="float: none;padding-top: 0px;padding-left: 0px;margin-left: auto;padding-right: 0px;margin-right: auto;border: 0px" border="0" alt="SQL Server 2016 Service Pack 1" src="https://msdnshared.blob.core.windows.net/media/2017/01/SQL-Server-2016-Service-Pack-1_thumb.png" width="910" height="504" /></a></p>
<h3>8. <a href="https://blogs.technet.microsoft.com/dataplatforminsider/2016/06/09/wideworldimporters-the-new-sql-server-sample-database/">WideWorldImporters: The new SQL Server sample database</a></h3>
<p>We figured a major release of the magnitude of SQL Server 2016 deserved its own new sample database. Besides, it was time for an upgrade of the erstwhile AdventureWorks database, which had a good run for over a decade. The new Wide World Importers sample illustrates how the rich feature set of SQL Server can be used in a real world setting. While it doesn’t attempt to use every single feature (as that would be unrealistic!) it does showcase a variety of key SQL Server 2016 capabilities and performance enhancements.</p>
<h3>7. <a href="https://blogs.technet.microsoft.com/dataplatforminsider/2016/01/11/preview-the-newest-odbc-sql-server-driver-for-windows-and-linux/">Preview the newest ODBC SQL Server Driver for Windows and Linux</a></h3>
<p>Early last year, we announced our community technology preview of Microsoft ODBC Driver 13 for SQL Server on Windows and Linux, supporting Ubuntu, RedHat and SUSE distributions, providing robust data access to Microsoft SQL Server and Microsoft Azure SQL Database via ODBC on Windows and Linux platforms.</p>
<h3>6. <a href="https://blogs.technet.microsoft.com/dataplatforminsider/2016/06/06/microsoft-announces-major-commitment-to-apache-spark/">Microsoft announces major commitment to Apache Spark</a></h3>
<p>Back in June last year, we announced an extensive commitment for Spark to power Microsoft’s big data and analytics offerings, including Cortana Intelligence, Power BI, and Microsoft R Server.</p>
<p><a href="https://msdnshared.blob.core.windows.net/media/2017/01/Spark.png"><img title="Spark" style="float: none;padding-top: 0px;padding-left: 0px;margin-left: auto;padding-right: 0px;margin-right: auto;border: 0px" border="0" alt="Spark" src="https://msdnshared.blob.core.windows.net/media/2017/01/Spark_thumb.png" width="128" height="67" class="aligncenter" /></a></p>
<h3>5. <a href="https://blogs.technet.microsoft.com/dataplatforminsider/2016/01/05/json-in-sql-server-2016-part-1-of-4/">JSON in SQL Server 2016: Part 1 of 4</a></h3>
<p>JSON is one of the most widely used data exchange formats, with most modern services returning information as JSON text, and web browsers using information formatted as JSON. JSON is the common format when semi-structured information needs to be stored, such as in files. Because so much data is formatted as JSON, it’s critical for SQL Server to process JSON text retrieved from other systems, and to format information retrieved from SQL Server tables as JSON text. This was the first post in a 4-part series; use the links that follow to read <a href="https://blogs.technet.microsoft.com/dataplatforminsider/2016/01/06/json-in-sql-server-2016-part-2-of-4/">Part 2</a>, <a href="https://blogs.technet.microsoft.com/dataplatforminsider/2016/01/07/json-in-sql-server-2016-part-3-of-4/">Part 3</a> and <a href="https://blogs.technet.microsoft.com/dataplatforminsider/2016/01/08/json-in-sql-server-2016-part-4-of-4/">Part 4</a> of the series.</p>
<h3>4. <a href="https://blogs.technet.microsoft.com/dataplatforminsider/2016/12/16/sql-server-on-linux-how-introduction/">SQL Server on Linux: How? Introduction</a></h3>
<p>Many of you in the community expressed your curiosity about how we made SQL Server run on Linux, so – just before the holidays – we kicked off a blog series last month to share technical details about this very topic, starting with an introduction to the journey of offering SQL Server on Linux.</p>
<h3>3. <a href="https://blogs.technet.microsoft.com/dataplatforminsider/2016/11/16/announcing-sql-server-on-linux-public-preview-first-preview-of-next-release-of-sql-server/">Announcing SQL Server on Linux public preview, first preview of next release of SQL Server</a></h3>
<p>The community shared our excitement when we announced the first public preview of SQL Server on Linux in November, allowing developers to create their SQL Server solutions on Linux, Windows, Docker, or macOS (via Docker), and then deploy to Linux, Windows, or Docker, on-premises or in the cloud. There’s been tremendous interest from large enterprises as well, with a majority of Fortune 500 companies signing up for this preview release.</p>
<p><a href="https://msdnshared.blob.core.windows.net/media/2017/01/Microsoft-Heart-Pengiun.png"><img title="Microsoft Heart Pengiun" style="float: none;padding-top: 0px;padding-left: 0px;margin-left: auto;padding-right: 0px;margin-right: auto;border: 0px" border="0" alt="Microsoft Heart Pengiun" src="https://msdnshared.blob.core.windows.net/media/2017/01/Microsoft-Heart-Pengiun_thumb.png" width="640" height="274" class="aligncenter" /></a></p>
<h3>2. <a href="https://blogs.technet.microsoft.com/dataplatforminsider/2016/06/01/sql-server-2016-is-generally-available-today/">SQL Server 2016 is generally available today</a></h3>
<p>Perhaps our biggest announcement of the year was the general availability of SQL Server 2016, back in June. There was also <a href="https://blogs.technet.microsoft.com/dataplatforminsider/2016/05/02/get-ready-sql-server-2016-coming-on-june-1st/">this post</a> a month prior to the release, recapping key capabilities and benchmarks we achieved. As the world’s fastest, most price-performant database for HTAP (Hybrid Transactional and Analytical Processing) with updateable, in-memory columnstores and advanced analytics via deep integration with R Services, SQL Server offers industry-leading TCO and is a truly watershed product.</p>
<p><a href="https://msdnshared.blob.core.windows.net/media/2017/01/SQL-Server-2016-leading-TCO.png"><img title="SQL Server 2016 leading TCO" style="float: none;padding-top: 0px;padding-left: 0px;margin-left: auto;padding-right: 0px;margin-right: auto;border: 0px" border="0" alt="SQL Server 2016 leading TCO" src="https://msdnshared.blob.core.windows.net/media/2017/01/SQL-Server-2016-leading-TCO_thumb.png" width="640" height="360" class="aligncenter" /></a></p>
<h3></h3>
<h3>1. <a href="https://blogs.technet.microsoft.com/dataplatforminsider/2016/03/31/microsoft-sql-server-developer-edition-is-now-free/">Microsoft SQL Server Developer Edition is now free</a></h3>
<p>The best things in life are free, as they say – and so it was certainly so for our announcement that SQL Server Developer Edition was being made a free download for all Visual Studio Dev Essentials members. Note that the SQL Server Developer Edition is only for development and testing, and not for production environments or for use with production data. This announcement was an important step in Microsoft making SQL Server ever more broadly accessible.</p>
<p>Stay tuned to this channel for much more exciting news during the year ahead about databases in general, and the SQL Server family in particular.</p>
<p>We wish all of our readers a very happy new year!<br />
SQL Server Blog Team</p>
]]></content:encoded>
</item>
<item>
<title>SQL Server next version Community Technology Preview 1.1 now available</title>
<link>https://blogs.technet.microsoft.com/dataplatforminsider/2016/12/16/sql-server-next-version-community-technology-preview-1-1-now-available/</link>
<comments>https://blogs.technet.microsoft.com/dataplatforminsider/2016/12/16/sql-server-next-version-community-technology-preview-1-1-now-available/#comments</comments>
<pubDate>Fri, 16 Dec 2016 20:30:51 +0000</pubDate>
<dc:creator><![CDATA[SQL Server Team]]></dc:creator>
<category><![CDATA[Uncategorized]]></category>
<guid isPermaLink="false">https://blogs.technet.microsoft.com/dataplatforminsider/?p=18365</guid>
<description><![CDATA[Microsoft is excited to announce that the next version of SQL Server (SQL Server v.Next) Community Technology Preview (CTP) 1.1 is now available for download on both Windows and Linux. In SQL Server v.Next CTP 1.1, part of our rapid preview model, we made enhancements to several database engine and Business Intelligence (BI) capabilities which... <a aria-label="read more about SQL Server next version Community Technology Preview 1.1 now available" href="https://blogs.technet.microsoft.com/dataplatforminsider/2016/12/16/sql-server-next-version-community-technology-preview-1-1-now-available/" class="read-more">Read more</a>]]></description>
<content:encoded><![CDATA[<p>Microsoft is excited to announce that the next version of SQL Server (SQL Server v.Next) Community Technology Preview (CTP) 1.1 is now available for download on both Windows and Linux. In SQL Server v.Next CTP 1.1, part of our rapid preview model, we made enhancements to several database engine and Business Intelligence (BI) capabilities which you can try in your development and test environments.</p>
<h2>Key enhancements in SQL Server v.Next on Windows CTP 1.1 for Analysis Services tabular models include:</h2>
<ul>
<li>New infrastructure for data connectivity and ingestion into tabular models with support for TOM APIs and TMSL scripting. This infrastructure enables:
<ul>
<li>Support for additional data sources, such as MySQL. Additional data sources are planned in upcoming CTPs.</li>
<li>Data transformation and data mashup capabilities.</li>
</ul>
</li>
<li>Support for BI tools such as Microsoft Excel enable drill-down to detailed data from an aggregated report. For example, when end-users view total sales for a region and month, they can view the associated order details.</li>
<li>Support for ragged hierarchies in reports, such as organizational and account charts.</li>
<li>Enhanced security for tabular models, including the ability to set permissions to help secure individual tables.</li>
</ul>
<p>For more detailed information about Analysis Services in SQL Server v.Next CTP 1.1, see the <a href="https://go.microsoft.com/fwlink/?linkid=837760">Analysis Services Team Blog</a>.</p>
<h2>Key SQL Server v.Next on Windows and Linux CTP 1.1 database engine enhancements include:</h2>
<ul>
<li>Language and performance enhancements to natively compiled T-SQL modules, including support for OPENJSON, FOR JSON, JSON built ins as well as memory-optimized tables support for computed columns.</li>
<li>Improved the performance of updates to non-clustered columnstore indexes in the case when the row is in the delta store.</li>
<li>Batch mode queries now support “memory grant feedback loops,” which learn from memory used during query execution and adjusts on subsequent query executions; this can allow more queries to run on systems that are otherwise blocking on memory.</li>
<li>New T-SQL language features:
<ul>
<li>Introducing three new string functions: TRIM, CONCAT_WS, and TRANSLATE</li>
<li>BULK IMPORT supports CSV format and Azure Blob Storage as file source</li>
<li>STRING_AGG supports WITHIN GROUP (ORDER BY)s</li>
</ul>
</li>
</ul>
<p>In addition, we have added support for Red Hat 7.3 and Ubuntu 16.10 to <b>SQL Server on Linux</b>.</p>
<p>For additional detail, please visit <a href="https://msdn.microsoft.com/en-US/library/mt788653(SQL.130).aspx">What’s New in SQL Server v.Next</a>, <a href="https://msdn.microsoft.com/en-US/library/mt788652(SQL.130).aspx">Release Notes</a> and <a href="https://docs.microsoft.com/en-us/sql/linux/">Linux documentation</a>.</p>
<h2>Download SQL Server v.Next CTP 1.1 preview today!</h2>
<p>SQL Server v.Next brings the power of SQL Server to both Windows – and for the first time ever – Linux. SQL Server enables developers and organizations to build intelligent applications with industry-leading performance and security technologies using their preferred language and environment.</p>
<p>Try the preview of the next release of SQL Server today! Get started with the preview of SQL Server on Linux, macOS (via Docker) and Windows with <a href="http://www.microsoft.com/sql-server/developer-get-started">our developer tutorials</a> that show you how to install and use SQL Server v.Next on macOS, Docker, Windows, RHEL and Ubuntu and quickly build an app in a programming language of your choice.</p>
<ul>
<li><a href="https://docs.microsoft.com/en-us/sql/linux/sql-server-linux-setup-red-hat">Install on Red Hat Enterprise Linux</a></li>
<li><a href="https://docs.microsoft.com/en-us/sql/linux/sql-server-linux-setup-ubuntu">Install on Ubuntu Linux</a></li>
<li><a href="https://docs.microsoft.com/en-us/sql/linux/">Pull and run a Docker container on Linux, Windows, or macOS</a></li>
<li><a href="https://www.microsoft.com/evalcenter/evaluate-sql-server-vnext-ctp">Download the preview for Windows</a></li>
<li><a href="https://azure.microsoft.com/en-us/marketplace/?term=%22SQL+Server+vNext%22">Create a SQL Server on Linux virtual machine in Azure</a></li>
<li>Install on SUSE Linux Enterprise Server (coming soon)</li>
</ul>
<p>Visit the <a href="http://www.sqlserveronlinux.com/">SQL Server v.Next webpage</a> to learn more. To experience the new, exciting features in SQL Server v.Next and our rapid release model, download the preview on Linux and Windows and start evaluating the impact these new innovations can have for your business. Have questions? Join the discussion of the new SQL Server v.Next capabilities at <a href="https://social.msdn.microsoft.com/Forums/sqlserver/en-US/home">MSDN</a>. If you run into an issue or would like to make a suggestion, you can let us know at <a href="https://connect.microsoft.com/SQLServer">Connect</a>. We look forward to hearing from you!</p>
]]></content:encoded>
<wfw:commentRss>https://blogs.technet.microsoft.com/dataplatforminsider/2016/12/16/sql-server-next-version-community-technology-preview-1-1-now-available/feed/</wfw:commentRss>
<slash:comments>3</slash:comments>
</item>
<item>
<title>SQL Server on Linux: How? Introduction</title>
<link>https://blogs.technet.microsoft.com/dataplatforminsider/2016/12/16/sql-server-on-linux-how-introduction/</link>
<comments>https://blogs.technet.microsoft.com/dataplatforminsider/2016/12/16/sql-server-on-linux-how-introduction/#comments</comments>
<pubDate>Fri, 16 Dec 2016 18:00:21 +0000</pubDate>
<dc:creator><![CDATA[SQL Server Team]]></dc:creator>
<category><![CDATA[Uncategorized]]></category>
<guid isPermaLink="false">https://blogs.technet.microsoft.com/dataplatforminsider/?p=18105</guid>
<description><![CDATA[This post was authored by Scott Konersmann, Partner Engineering Manager, SQL Server, Slava Oks, Partner Group Engineering Manager, SQL Server, and Tobias Ternstrom, Principal Program Manager, SQL Server. Introduction We first announced SQL Server on Linux in March, and recently released the first public preview of SQL Server on Linux (SQL Server v.Next CTP1) at... <a aria-label="read more about SQL Server on Linux: How? Introduction" href="https://blogs.technet.microsoft.com/dataplatforminsider/2016/12/16/sql-server-on-linux-how-introduction/" class="read-more">Read more</a>]]></description>
<content:encoded><![CDATA[<p><em>This post was authored by Scott Konersmann, Partner Engineering Manager, SQL Server, Slava Oks, Partner Group Engineering Manager, SQL Server, and Tobias Ternstrom, Principal Program Manager, SQL Server.</em></p>
<h4><a><span style="color: #000000">Introduction</span></a></h4>
<p>We <a href="http://blogs.microsoft.com/blog/2016/03/07/announcing-sql-server-on-linux/#sm.00000g5feswlpncs5vljgonb8xzu0">first announced</a> SQL Server on Linux in March, and recently released the <a href="https://blogs.technet.microsoft.com/dataplatforminsider/2016/11/16/announcing-sql-server-on-linux-public-preview-first-preview-of-next-release-of-sql-server/">first public preview of SQL Server on Linux</a> (SQL Server v.Next CTP1) at the Microsoft Connect(); conference. We’ve been pleased to see the positive reaction from our customers and the community; in the two weeks following the release, there were more than 21,000 downloads of the preview. A lot of you are curious to hear more about how we made SQL Server run on Linux (and some of you have already figured out and posted interesting articles about part of the story with “Drawbridge”). We decided to kick off a blog series to share technical details about this very topic starting with an introduction to the journey of offering SQL Server on Linux. Hopefully you will find it as interesting as we do! J</p>
<h4>Summary</h4>
<p>Making SQL Server run on Linux involves introducing what is known as a Platform Abstraction Layer (“PAL”) into SQL Server. This layer is used to align all operating system or platform specific code in one place and allow the rest of the codebase to stay operating system agnostic. Because of SQL Server’s long history on a single operating system, Windows, it never needed a PAL. In fact, the SQL Server database engine codebase has many references to libraries that are popular on Windows to provide various functionality. In bringing SQL Server to Linux, we set strict requirements for ourselves to bring the full functional, performance, and scale value of the SQL Server RDBMS to Linux. This includes the ability for an application that works great on SQL Server on Windows to work equally great against SQL Server on Linux. Given these requirements and the fact that the existing SQL Server OS dependencies would make it very hard to provide a highly capable version of SQL Server outside of Windows in reasonable time it was decided to marry parts of the Microsoft Research (MSR) project <a href="https://www.microsoft.com/en-us/research/project/drawbridge/">Drawbridge</a> with SQL Server’s existing platform layer <a href="https://blogs.msdn.microsoft.com/slavao/2005/07/20/platform-layer-for-sql-server/">SQL Server Operating System</a> (SOS) to create what we call the SQLPAL. The Drawbridge project provided an abstraction between the underlying operating system and the application for the purposes of secure containers and SOS provided robust memory management, thread scheduling, and IO services. Creating SQLPAL enabled the existing Windows dependencies to be used on Linux with the help of parts of the Drawbridge design focused on OS abstraction while leaving the key OS services to SOS. We are also changing the SQL Server database engine code to by-pass the Windows libraries and call directly into SQLPAL for resource intensive functionality.</p>
<h4><a><span style="color: #000000">Requirements for supporting Linux</span></a></h4>
<p>SQL Server is Microsoft’s flagship database product which with close to 30 years of development behind it. At a high level, the list below represents our requirements as we designed the solution to make the SQL Server RDBMS available on multiple platforms:</p>
<ol>
<li>Quality and security must meet the same high bar we set for SQL Server on Windows</li>
<li>Provide the same value, both in terms of functionality, performance, and scale</li>
<li>Application compatibility between SQL Server on Windows and Linux</li>
<li>Enable a continued fast pace of innovation in the SQL Server code base and make sure new features and fixes appear immediately across platforms</li>
<li>Put in place a foundation for future SQL Server suite services (such as Integration Services) to come to Linux</li>
</ol>
<p>To make SQL Server support multiple platforms, the engineering task is essentially to remove or abstract away its dependencies on Windows. As you can imagine, after decades of development against a single operating system, there are plenty of OS-specific dependencies across the code base. In addition, the code base is huge. There are tens of millions of lines of code in SQL Server.</p>
<p>SQL Server depends on various libraries and their functions and semantics commonly used in Windows development that fall into three categories:</p>
<ul>
<li>“Win32” (ex. user32.dll)</li>
<li>NT Kernel (ntdll.dll)</li>
<li>Windows application libraries (such as MSXML)</li>
</ul>
<p>You can think of these as core library functions, most of them have nothing to do with the operating system kernel and only execute in user mode.</p>
<p>While SQL Server has dependencies on both Win32 and the Windows kernel, the most complex dependency is that of Windows application libraries that have been added over the years in order to provide new functionality. Here are some examples:</p>
<ul>
<li>SQL Server’s XML support uses <a href="https://en.wikipedia.org/wiki/MSXML">MSXML</a> which is used to parse and process XML documents within SQL Server.</li>
<li>SQLCLR hosts the Common Language Runtime (<a href="https://en.wikipedia.org/wiki/Common_Language_Runtime">CLR</a>) for both system types as well as user defined types and CLR stored procedures.</li>
<li>SQL Server has some components written in <a href="https://en.wikipedia.org/wiki/Component_Object_Model">COM</a> like the <a href="https://www.microsoft.com/en-us/download/details.aspx?id=17282">VDI</a> interface for backups.</li>
<li>Heterogeneous distributed transactions are controlled through Microsoft Distributed Transaction Coordinator (MS <a href="https://en.wikipedia.org/wiki/Microsoft_Distributed_Transaction_Coordinator">DTC</a>)</li>
<li>SQL Server Agent integrates with many Windows subsystems (shell execution, Windows Event Log, SMTP Mail, etc.).</li>
</ul>
<p>These dependencies are the biggest challenge for us to overcome to meet our goals of bringing the same value and having a very high level compatibility between SQL Server on Windows and Linux. As an example, to re-implement something like SQLXML would take a significant amount of time and would run a high risk of not providing the same semantics as before, and could potentially break applications. The option of completely removing these dependencies would mean we must also remove the functionality they provide from SQL Server on Linux. If the dependencies were edge cases and only impacting very few customer visible features, we could have considered it. As it turns out, removing them would cause us to have to remove tons of features from SQL Server on Linux which would go against our goals around compatibility and value across operating systems.</p>
<p>We could take the approach of doing this re-implementation piecemeal, bringing value little by little. While this would be possible, it would also go against the requirements because it would mean that there would be a significant gap between SQL Server on Linux and Windows for years. The resolution lies in the right platform abstraction layer.</p>
<h2>Building a PAL</h2>
<p>Software that is supported across multiple operating systems always has an implementation of some sort of Platform Abstraction Layer (PAL). The PAL layer is responsible for abstraction of the calls and semantics of the underlying operating system and its libraries from the software itself. The next couple of sections consider some of the technology that we investigated as solutions to building a PAL for SQL Server.</p>
<h4>SQL Operating System (SOS or SQLOS)</h4>
<p>In the SQL Server 2005 release, a platform layer was created between the SQL Server engine and Windows called the SQL Operating System (SOS). This layer was responsible for user mode thread scheduling, memory management, and synchronization (see <a href="https://blogs.msdn.microsoft.com/slavao/2005/07/20/platform-layer-for-sql-server/">SQLOS</a> for reference). A key reason for the creation of SOS was that it allowed for a centralized set of low level management and diagnostics functionality to be provided to customers and support (subset of Dynamic Management Views/DMVs and Extended Events/XEvents). This layer allowed us to minimize the number of system calls involved in scheduling execution by running non-preemptively and letting SQL Server do its own resource management. While SOS improved performance and greatly helped supportability and debugging, it did not provide a proper abstraction layer from the OS dependencies described above, i.e. Windows semantics were carried through SOS and exposed to the database engine.</p>
<p><a href="https://msdnshared.blob.core.windows.net/media/2016/12/012.jpg"><img width="939" height="630" title="01" class="aligncenter" style="float: none;padding-top: 0px;padding-left: 0px;margin-left: auto;padding-right: 0px;margin-right: auto;border: 0px" alt="01" src="https://msdnshared.blob.core.windows.net/media/2016/12/01_thumb.jpg" border="0" /></a></p>
<p>In the scenario where we would completely remove the dependencies on the underlying operating system from the database engine, the best option was to grow SOS into a proper Platform Abstraction Layer (PAL). <b>All</b> the calls to Windows APIs would be routed through a new set of equivalent APIs in SOS and a new host extension layer would be added on the bottom of SOS that would interact with the operating system. While this would resolve the system call dependencies, it would not help with the dependencies on the higher-level libraries.</p>
<h4><a><span style="color: #000000">Drawbridge</span></a></h4>
<p>Drawbridge was an Microsoft Research project (see <a href="https://www.microsoft.com/en-us/research/project/drawbridge/">Drawbridge</a> for reference) that focused on drastically reducing the virtualization resource overhead incurred when hosting many Virtual Machines on the same hardware. The research involved two ideas. The first idea was a “picoprocess” which consists of an empty address space, a monitor process that interacts with the host operating system on behalf of the picoprocess, and a kernel driver that allows a driver to populate the address space at startup and implements a host Application Binary Interface (ABI) that allows the picoprocess to interact with the host. The second idea was a user mode Library OS, sometimes referred to as LibOS. Drawbridge provided a working Windows Library OS that could be used to run Windows programs on a Windows host. This Library OS implements a subset of the 1500+ Win32 and NT ABIs and stubs the rest to either succeed or fail depending on the type of call.</p>
<p><a href="https://msdnshared.blob.core.windows.net/media/2016/12/021.png"><img width="700" height="930" title="02" class="aligncenter" style="float: none;padding-top: 0px;padding-left: 0px;margin-left: auto;padding-right: 0px;margin-right: auto;border: 0px" alt="02" src="https://msdnshared.blob.core.windows.net/media/2016/12/02_thumb.png" border="0" /></a></p>
<p>Our needs didn’t align with the original goals of the Drawbridge research. For instance, the picoprocess idea isn’t something needed for moving SQL Server to other platforms. However, there were a couple of synergies that stood out:</p>
<ol>
<li>Library OS implemented most of the 1500+ Windows ABIs in user mode and only 45-50 ABIs were needed to interact with the host. These ABIs were for address space and memory management, host synchronization, and IO (network and disk). This made for a very small surface area that needs to be implemented to interact with a host. That is extremely attractive from a platform abstraction perspective.</li>
<li>Library OS was capable of hosting other Windows components. Enough of the Win32 and NT layers were implemented to host CLR, MSXML, and other APIs that the SQL suite depends on. This meant that we could get more functionality to work without rewriting whole features.</li>
</ol>
<p>There were also some risk and reward tradeoffs:</p>
<ol>
<li>The Microsoft Research project was complete and there was no support for Drawbridge. Therefore, we needed to take a source snapshot and modify the code for our purposes. The risks were around the costs to ramp up a team on the Library OS, modify it to be suitable for SQL Server, and make it perform comparably with Windows. On the positive side, this would mean everything is in user mode and we would own all the code within the stack. Performance critical code can be optimized because we can modify all layers of the stack including SQL Server, the Library OS, and the host interface as needed to make SQL Server perform. Since there are no real boundaries in the process, it is possible for SQL Server to call Linux.</li>
<li>The original Drawbridge project was built on Windows and used a kernel driver and monitor process. This would need to be dropped in favor of a user mode only architecture. In the new architecture, the host extension (referred to as PAL in the Drawbridge design) on Windows would move from a kernel driver to just a user mode program. Interestingly enough, one of the researchers had developed a rough prototype for Linux that proved it could be done.</li>
<li>Because the technologies were created independently there was a large amount of overlapping functionality. SOS had subsystems for object management, memory management, threading/scheduling, synchronization, and IO (disk and network). The Library OS and Host Extension also had similar functionality. These systems would need to be rationalized down to a single implementation.</li>
</ol>
<table align="center" border="0" cellspacing="0" cellpadding="0">
<tbody>
<tr>
<td width="161" valign="top"><b>Technologies</b></td>
<td width="144" valign="top">
<p align="center"><b>SOS</b></p>
</td>
<td width="156" valign="top">
<p align="center"><b>Library OS</b></p>
</td>
<td width="160" valign="top">
<p align="center"><b>Host Extension</b></p>
</td>
</tr>
<tr>
<td width="161" valign="top">
<p align="left">Object Management</p>
</td>
<td width="144" valign="top">
<p align="center">✔</p>
</td>
<td width="156" valign="top">
<p align="center">✔</p>
</td>
<td width="160" valign="top">
<p align="center">✔</p>
</td>
</tr>
<tr>
<td width="161" valign="top">
<p align="left">Memory Management</p>
</td>
<td width="144" valign="top">
<p align="center">✔</p>
</td>
<td width="156" valign="top">
<p align="center">✔</p>
</td>
<td width="160" valign="top">
<p align="center">✔</p>
</td>
</tr>
<tr>
<td width="161" valign="top">
<p align="left">Threading/Scheduling</p>
</td>
<td width="144" valign="top">
<p align="center">✔</p>
</td>
<td width="156" valign="top">
<p align="center">✔</p>
</td>
<td width="160" valign="top">
<p align="center">✔</p>
</td>
</tr>
<tr>
<td width="161" valign="top">
<p align="left">Synchronization</p>
</td>
<td width="144" valign="top">
<p align="center">✔</p>
</td>
<td width="156" valign="top">
<p align="center">✔</p>
</td>
<td width="160" valign="top">
<p align="center">✔</p>
</td>
</tr>
<tr>
<td width="161" valign="top">
<p align="left">I/O (Disk, Network)</p>
</td>
<td width="144" valign="top">
<p align="center">✔</p>
</td>
<td width="156" valign="top">
<p align="center">✔</p>
</td>
<td width="160" valign="top">
<p align="center">✔</p>
</td>
</tr>
</tbody>
</table>
<p> </p>
<h4><a></a><a><span style="color: #000000">Meet </span></a><span style="color: #000000">SQLPAL</span></h4>
<p>As a result of the investigation, we decided on a hybrid strategy. We would merge SOS and Library OS from Drawbridge to create the SQL PAL (SQL Platform Abstraction Layer). For areas of Library OS that SQL Server does not need, we would remove them. To merge these architectures, changes were needed in all layers of the stack.</p>
<p>The new architecture consists of a set of SOS direct APIs which don’t go through any Win32 or NT syscalls. For code without SOS direct APIs they will either go through a hosted Windows API (like MSXML) or NTUM (NT User Mode API – this is the 1500+ Win32 and NT syscalls). All the subsystems like storage, network, or resource management will be based on SOS and will be shared between SOS direct and NTUM APIs.</p>
<p><a href="https://msdnshared.blob.core.windows.net/media/2016/12/031.png"><img width="1141" height="545" title="03" class="aligncenter" style="float: none;padding-top: 0px;padding-left: 0px;margin-left: auto;padding-right: 0px;margin-right: auto;border: 0px" alt="03" src="https://msdnshared.blob.core.windows.net/media/2016/12/03_thumb.png" border="0" /></a></p>
<p>This architecture provides some interesting characteristics:</p>
<ul>
<li>Everything running in process boils down to the same platform assembly code. The CPU can’t tell the difference between the code that is providing Win32 functionality to SQL Server or native Linux code.</li>
<li>Even though the architecture shows layering, there are no real boundaries within the process (<a href="https://en.wikipedia.org/wiki/The_Matrix">There is no spoon!</a>). If code running in SQL Server which is performance critical needs to call Linux it can do that directly with a very small amount of assembler via the SOS direct APIs to setup the stack correctly and process the result. An example where this has been done is the disk IO path. There is a small amount of conversion code left to convert from Windows scatter/gather input structure to Linux vectored IO structure. Other disk IO types don’t require any conversions or allocations.</li>
<li>All resources in the process can be managed by SQLPAL. In SQL Server, before SQLPAL, most resources such as memory and threads were controlled, but there were some things outside it’s control. Some libraries and Win32/NT APIs would create threads on their own and do memory allocations without using the SOS APIs. With this new architecture, even the Win32 and NT APIs would be based on SQLPAL so every memory allocation and thread would be controlled by SQL PAL. As you can see this also benefits SQL Server on Windows.</li>
<li><a><span style="color: #000000">For SQL Server on Linux we are using about 81 MB of uncompressed Windows libraries, so it’s a tiny fraction (less than 1%) of a typical Windows installation. SQLPAL itself is currently around 8 MB.</span></a></li>
</ul>
<h4>Process Model</h4>
<p>The following diagram shows what the address space looks like when running. The host extension is simply a native Linux application. When host extension starts it loads and initializes SQLPAL, SQLPAL then brings up SQL Server. SQLPAL can launch software isolated processes that are simply a collection of threads and allocations running within the same address space. We use that for things like SQLDumper which is an application that is run when SQL Server encounters a problem to collect an enlightened crash dump.</p>
<p>One point to reiterate is that even though this might look like a lot of layers there aren’t any hard boundaries between SQL Server and the host.</p>
<p><a href="https://msdnshared.blob.core.windows.net/media/2016/12/041.png"><img width="467" height="712" title="04" class="aligncenter" style="float: none;padding-top: 0px;padding-left: 0px;margin-left: auto;padding-right: 0px;margin-right: auto;border: 0px" alt="04" src="https://msdnshared.blob.core.windows.net/media/2016/12/04_thumb.png" border="0" /></a></p>
<h4><a><span style="color: #000000">Evolution of SQLPAL</span></a></h4>
<p>At the start of the project, SQL Server was built on SOS and Library OS was independent. The eventual goal is to have a merged SOS and Library OS as the core of SQL PAL. For public preview, this merge wasn’t fully completed, but the heart of SQLPAL had been replaced with SOS. For example, threads and memory already use SOS functionality instead of the original Drawbridge implementations.</p>
<p>The result is that there are two instances of SOS running inside the CTP1 release: one in SQL Server and one in SQLPAL . This works fine because the SOS instance in SQL Server is still using Win32 APIs which call down into the SQLPAL. The SQLPAL instance of the SOS code has been changed to call the host extension ABIs (i.e. the native Linux code) instead of Win32.</p>
<p>Now we are working on removing the SOS instance from SQL Server. We are exposing the SOS APIs from the SQLPAL. Once this is completed everything will flow through the single SQLPAL SOS instance.</p>
<h2>More posts</h2>
<p>We are planning more of these posts to share to tell you about our journey, which we believe has been amazing and a ton of fun worth sharing. Please provide comments if there are specific areas you are interested in us covering!</p>
<p>Thanks!</p>
]]></content:encoded>
<wfw:commentRss>https://blogs.technet.microsoft.com/dataplatforminsider/2016/12/16/sql-server-on-linux-how-introduction/feed/</wfw:commentRss>
<slash:comments>12</slash:comments>
</item>
<item>
<title>SQL Server + C#: What’s new</title>
<link>https://blogs.technet.microsoft.com/dataplatforminsider/2016/12/15/sql-server-c-whats-new/</link>
<pubDate>Thu, 15 Dec 2016 17:00:11 +0000</pubDate>
<dc:creator><![CDATA[SQL Server Team]]></dc:creator>
<category><![CDATA[Uncategorized]]></category>
<guid isPermaLink="false">https://blogs.technet.microsoft.com/dataplatforminsider/?p=18355</guid>
<description><![CDATA[This post was authored by Andrea Lam, Program Manager, SQL Server With the release of SQL Server v.Next public preview on Linux and Windows, the ability to connect to SQL Server on Linux, Windows, Docker or macOS (via Docker) makes cross-platform support for all connectors, including .NET Framework and .NET Core SqlClient, even more important.... <a aria-label="read more about SQL Server + C#: What’s new" href="https://blogs.technet.microsoft.com/dataplatforminsider/2016/12/15/sql-server-c-whats-new/" class="read-more">Read more</a>]]></description>
<content:encoded><![CDATA[<p>This post was authored by Andrea Lam, Program Manager, SQL Server</p>
<p><iframe width="960" height="540" src="https://channel9.msdn.com/Events/Connect/2016/151/player" frameborder="0" allowfullscreen="allowfullscreen"></iframe></p>
<p>With the release of <a href="https://blogs.technet.microsoft.com/dataplatforminsider/2016/11/16/announcing-sql-server-on-linux-public-preview-first-preview-of-next-release-of-sql-server/">SQL Server v.Next public preview on Linux and Windows</a>, the ability to connect to SQL Server on Linux, Windows, Docker or macOS (via Docker) makes cross-platform support for all connectors, including .NET Framework and .NET Core SqlClient, even more important. To enable C# developers to use the newest SQL Server features, we have been updating SqlClient with client-side support for new features.</p>
<p>In .NET Framework, we have provided client-side support for Always Encrypted and added Azure Active Directory as an authentication method. We’ve also added a new connection string parameter called “Pool Blocking Period.” It can be used to select the behavior of the <a href="https://msdn.microsoft.com/en-us/library/8xx3tyca(v=vs.110).aspx">blocking period</a> when connecting to Azure SQL Database and SQL Server. Many applications that connect to Azure SQL Database need to render quickly, and the exponential blocking period can be problematic, especially when throwing errors. By adding the Pool Blocking Period parameter, we aim to improve the experience for connections to Azure SQL Database. Learn more about the new parameter <a href="https://msdn.microsoft.com/en-us/library/system.data.sqlclient.sqlconnectionstringbuilder.poolblockingperiod(v=vs.110).aspx">here</a>.</p>
<p><a href="https://github.com/dotnet/core">.NET Core</a> is the cross-platform, open-source implementation of the .NET Framework. The project includes <a href="https://github.com/dotnet/corefx">CoreFX</a>, the foundational libraries of .NET Core. Cross-platform support allows developers to seamlessly run applications on their operating system of choice, regardless of the platform it was developed on. For example, an app developed on Windows can be deployed to macOS and Linux, without ever having to port any code. To connect apps written in .NET Core to SQL Server (hosted anywhere), developers can use System.Data.SqlClient available in CoreFX. By developing in .NET Core on GitHub, we have been able to get feedback quickly and are actively working to enable the breadth of application scenarios and workloads.</p>
<h4>Get started today</h4>
<ul>
<li>Check out <a href="https://github.com/dotnet/corefx">CoreFX on GitHub</a>! Make pull requests and let us know what you think.</li>
<li>Try the new <a href="http://aka.ms/sqldev?utm_medium=Referral&utm_source=https://blogs.technet.microsoft.com/dataplatforminsider/2016/12/14/sql-server-c-whats-new/">getting started tutorials</a> that show you how to:
<ul>
<li>Install SQL Server on Linux/macOS/Docker/Windows</li>
<li>Create a simple app using C# and other popular programming languages with SQL Server</li>
<li>Create a simple app using popular web frameworks and Object Relational Mapping (ORM) frameworks with SQL Server</li>
<li>Try out some cool SQL Server features that can make your apps shine</li>
</ul>
</li>
</ul>
<h4>Connect with us</h4>
<ul>
<li>Join the conversation at <a href="https://gitter.im/mssqldev/Lobby">https://gitter.im/mssqldev</a>.</li>
<li><a href="https://helsinki.catalysis.com/registration.aspx">Sign up to stay informed</a> about new SQL Server on Linux developments.</li>
</ul>
<h4>Learn more</h4>
<ul>
<li>Visit the <a href="https://channel9.msdn.com/Events/Connect/2016">Connect(); webpage</a> to watch overview, security, high-availability and developer tools videos about SQL Server on Linux on demand.</li>
<li>Visit the <a href="http://www.sqlserveronlinux.com/">webpage for the next release of SQL Server</a>.</li>
</ul>
<h4>Other videos in this series</h4>
<ul>
<li><a href="https://channel9.msdn.com/events/connect/2016/160">SQL Server + Node.js: what’s new</a></li>
<li><a href="https://channel9.msdn.com/events/connect/2016/152">SQL Server + Java: what’s new</a></li>
<li><a href="https://channel9.msdn.com/events/connect/2016/182">SQL Server + PHP: what’s new</a></li>
<li><a href="https://channel9.msdn.com/events/connect/2016/189">SQL Server + Python: what’s new</a></li>
<li>Watch the <a href="https://www.youtube.com/watch?v=oEpJB87Xg9U&feature=youtu.be">Microsoft Mechanics video</a> to see how to get started with SQL Server on Linux in less than a minute.</li>
<li><a href="https://channel9.msdn.com/Events/Connect/2016/173/">SQL Server on Linux: use SQL Tools with SQL Server</a></li>
</ul>
]]></content:encoded>
</item>
</channel>
</rss>