Microsoft BI Image X: SQL 2012 RC0 Available for Download

Periodically Microsoft refreshes their BI Demo virtual image that contains a fully configured instance of their latest BI tools with demo data.  Last week they made the latest version available for download.  They are calling it BI Image X and it contains SQL 2012 RC0 and it is available here.  This one contains the new PowerView tool, as well as SharePoint and PowerPivot.  You will need a minimum of 8GB of RAM allocated to the VM.

Here’s the official description:

The Base ImageX Server is a virtual machine that can be hosted in Hyper-V that allows a user to test out the latest Business Intelligence features of SQL Server 2012 – RC0. The virtual machine has been configured to support the use of PowerView reports, PowerPivot Excel documents and various content packs allowing the user to explore the capabilities of SQL Server 2012. Working with the demos that can be installed on the ImageX server the user will have a clear picture of how the SQL Server 2012, Office and SharePoint tools support and enhance the self-service business intelligence environment.

**If you need to convert to VMWare to run on your dev machine (How come Windows 7 still  doesn’t support 64-bit VMs?), you can use VMWare vCenter Convert to convert the virtual hard disk to VMware.

Integration Services EzAPI Updated to Support SQL 2012

Just got word from Matt Masson on the SSIS development team that EzAPI has been updated to support SQL 2012.  This is greate news!  I have been using EzAPI for the last couple of months and I am very satisfied with it as an API for SSIS that allows me to focus on solving the problems I need to solve rather than getting hung up on the nuances of the SSIS COM objects and and wrappers.  If I need access to something that is not supported by EzAPI (Package Configurations, for example), I can just cast as a Package object and use the standard API and convert back to EzPackage when I finish.

Get the bits here (note, looks like you will need to compile the DLL yourself).  For now, click the link to download Change Set 86517.

**UPDATE**

Matt Masson posted the change list to the SSIS Team Blog:

Changes

  • Task and Component IDs updated for SQL 2012
    • For 2005/2008 support, please use a previous version of the code
  • Project upgraded to .NET 4
  • EzProject – SSIS 2012 Project object model support
  • Project Connection Manager support
  • EzScript – Script Task support
  • EzSrcDestMultiStreamPackage – new package template
  • EzLookup – new TreatDuplicateKeysAsError property
  • EzFileSystemTask – fixed multiple properties
  • EzActiveXScript – Removed as it is no longer supported in SQL 2012

Also, it looks like this release only works against 2012.  If you need SQL 2008, you will need the older version.

Index of all SQL Server build numbers since 7.0 Beta

Stumbled across a nice SQL Server build number site a while back that has a comprehensive (I think?) list of every SQL Server build from the SQL 7 beta 3 up through SQL 2012 RC1 including every CU and hotfix along the way.  There is also a link to download the KB that corresponds to each build if appropriate.  If you happen to know of a build that isn’t listed, it looks like this site is somewhat crowd-sourced, so post the details in the comments and it will likely make it up to the main page.

(If you look in the comments, you can even find little “anonymous” nuggets, such as the following: “SQL2012 RTM is 11.0.2100.19”)

#Meme15 Assignment #3 – Why and how Jason uses LinkedIn?

#Meme15

I saw Jason Strate’s (Blog|Twitter) recent call out for the February #Meme15 assignment, “Why and how you use LinkedIn?”, so I figured I’d chime in.

“Why I use LinkedIn?’ Simply put, it is my online resume.  While I try to update an actual resume in MS Word, having my information on LinkedIn allows me one online location where the content is saved.  But in a day and age where job security is not always dependent on what you know, I myself try to keep my MS Word resume up to date, and simply have LinkedIn mirror it.

As far as “How I use LinkedIn”, I’ll take this moment to say that I do not use it to connect to every person I know of or read about or even tried to get a job from.  On that last comment, that means I always decline accepting connections from random recruiters or even other SQL Server Professionals with whom I have never talked to or even corresponded to.  It’s not that I am being an elitist or such; it’s just that if I haven’t talked with you or met you in person, I simply won’t connect.  To me, this ensures my connections are genuine.  Each and every connection made, I feel I have worked with them in some way or another, and usually there’s a back story to relay in regards to that connection.  Whether it’s former co-workers from my days at Humana, to my early SQL Server reports and applications work at WaMu or Citi, to even recent connections via SQL Saturdays or other SQL Server Community events, the people I connect to, I know.

Anyway, that’s my take on Jason Strate’s call for the #Meme15.  There is definitely more to LinkedIn, and opportunities abound within its network of connections.  But for now, this is why and how I use LinkedIn.

My two and a half cents…

101 LINQ Samples Published by Microsoft

While trying to work through various problems in LINQ that I could easily solve in T-SQL, I found a sample at code.msdn.microsoft.com (I hadn’t seen that sight before, but seems like a potential gold mine).  It is called 101 LINQ Samples and at the time of this writing is the 3rd most popular download on the site.  Looking forward to digging into it.  Here’s the link to the main page with downloads.

Here are the individual links/list of topics:

Restriction Operators

Projection Operators

Partitioning Operators

Ordering Operators

Grouping Operators

Set Operators

Conversion Operators

Element Operators

Generation Operators

Quantifiers

Aggregate Operators

Miscellaneous Operators

Custom Sequence Operators

Query Execution

Join Operators

Lync 2010 Web Presentations and Video Scaling

One of the biggest frustrations with Microsoft Live Meeting 2005/2007 is that there is no scaling built in to the screen share capabilities.  If the presenter is running 1920×1080 and clients are running 1280×720 the presenter is not notified, but the viewers will not see the entire screen and instead have to scroll.

image

We are in the process of transitioning to Office 365, and one of the first features of Lync 2010 that I wanted to confirm was that Lync supported client screen scaling.  I am happy to report that it does, and it works perfectly!  To the right is a screenshot of a meeting shrunken as far as possible.  Some other very cool screen sharing features are available as well, including sharing (and scaling) multiple screens.  I have 3 screens in my development environment and I can choose to show any screen, any application running on a screen, or all 3 screens (2 are 1080p and 1 is 1600×1200 and it was not a problem).  You also can pull up a video thumbnail of what the clients are seeing, or even share a PowerPoint deck natively (not just sharing the PowerPoint app, but the .pptx file itself).  Lync 2010 is a great step forward over Live Meeting.

A Better Starting Point for .Net Console Applications

Often when prototyping .Net applications, I end up starting with a console app (as things get more complicated or require interactivity, I move to Windows forms).  It takes about 1 minute to get up and running with a console app. . . unless you want to start passing fancy parameters and things like that.  It doesn’t take much more effort, but the point is that you don’t want to re-invent the console app code, you want to spend your time prototyping.

Sometimes your production application might require a console app.  In my case, one portion of the application I am working on will need to be implemented as a console app (for now) so that it can be executed from a SQL Server Agent job.  I want the console app to be robust and production-ready, very different from the prototype apps I cobble together in the span of a couple of minutes.

image

In support of a more robust console application, I found a blog post with a great template put together by Alois Kraus.  There is a blog post covering the basics, and the template can be downloaded from the Visual Studio Gallery.

At a high level, the custom console template starts you off in a Cadillac instead of a bicycle when building a console application.  Right out of the gate you get a nice command line parser, help output, color console and much more robust error handling.  I doubt I will start with the basic console template again.

Using an Entity Framework Data Model in Multiple Projects

I’m starting to dig into Entity Framework 4 as part of a project that I am working on.  It is great, much easier than the old school data access methods in terms of getting up and running (it takes about 3 minutes).  There are a lot of articles out there, but I have still had to figure out a few less-obvious things along the way.

As part of getting my app ready for production, I am moving the data access layer to its own project.  I started out by creating a new project in my solution and then within that project I created a new entity data model.  After that, I just added the entity model project as a reference on my sample app project.  I was missing some references, so figured out I needed to add the following using statements:

using System.Data.Objects;
using System.Data.Objects.DataClasses;

After this I was a lot closer, able to compile the app but no data was being returned.  After a bit of searching, this article helped me a ton.  The only other thing I was missing was the connection string, which I didn’t even know existed partly due to the fact that Entity Framework is so easy to get up and running via the wizard, which hides this from you.  After I copied the connection string directly from Entity Model project, I just pasted it into the <configuration> section of the App.Config of my sample app and it worked perfectly right away.

  <connectionStrings>
    <add name="MyEntities" connectionString="metadata=res://*/MyModel.csdl|res://*/MyModel.ssdl|res://*/MyModel.msl;provider=System.Data.SqlClient;provider connection string=&quot;data source=(local);initial catalog=MyDB;integrated security=True;multipleactiveresultsets=True;App=EntityFramework&quot;" providerName="System.Data.EntityClient" />
  </connectionStrings>

Editing Existing SSIS Package via EzAPI or Standard SSIS API Doesn’t Update Layout in BIDS

As part of a project I am working on that requires me to programmatically alter existing packages, I ran into an annoyance (bug?) in SSIS that makes it hard to open packages after altering them.

The Problem

During an exchange on Codeplex forums with Matt Masson from the SSIS team, he summarized the workflow as follows:

  1. Created the package in Visual Studio
  2. Added stuff to it
  3. Saved it out to disk
  4. Opened and modified it with EzAPI

When I follow this workflow to add 10 connection managers, sequencimagee containers and execute package tasks programmatically to an existing package with nothing in the design surface, you will end up with a package that will show connection managers, but no sequence containers or execute package tasks.

As you can see in the screenshot to the right, there are a lot of connection managers visible, but no accompanying sequence container or execute package task.

If you looimagek at the Package Explorer tab, you will see that all of the “missing” sequence containers and execute package tasks were actually created and do exist in the package. You can see this in the Package Explorer in the screenshot to the left.  Additionally, if you execute the package from BIDS, programmatically added components appear in the output, just no green boxes or row counts in the Control Flow or Data Flow.

 

 

 

The Solution

Based on Matt Masson’s response to my question posted in the discussion area for EzAPI on Codeplex, I started digging into the XML layout code that is included at the end of the .dtsx package.  I found a blog post from the SQL Server Forum Support team that actually identified the problem I described above, and a solution.  Essentially the problem is that when a package is created in BIDS, the layout is created and stored inside a tag called <DTS:PackageVariable>.  When you modify a package programatically, this code is not updated since it is “BIDS generated code”.  When you open a package in BIDS after it has been altered, BIDS sees the existing layout code and doesn’t bother to parse the rest of the package to check for any missing objects for which layout code has not yet been created.

This problem does not occur on packages that are created programmatically from scratch because BIDS is able to detect that no layout code exists and will generate all of it the first time the package is opened and perform an auto-layout.

I adapted the code found at the SQL Server Forum Support Team FAQ to EzAPI with no additional problems.  One thing that I found is that there can be multiple <DTS:PackageVariable> sections, and IDS will not generate layout code until all of them have been removed.  To that end, my code uses Regex to get a count of how many code blocks will need to be purged.  Here is a sample of my adapted code (original is at the blog post linked above):

//Save the package object to XML
string strXML = null;
strXML = TestPackage.SaveToXML();

//Count instances of existing SSIS layout code in package.
int LayoutCount = Regex.Matches(strXML, "<DTS:PackageVariable>").Count;

//Remove the layout information.
for (int i = 0; i < LayoutCount; i++)
{
    strXML = strXML.Remove(strXML.IndexOf("<DTS:PackageVariable>"), strXML.IndexOf("</DTS:PackageVariable>") - strXML.IndexOf("<DTS:PackageVariable>") + 22);
}

Based on comments that Matt made, I am hopeful that this has been fixed in SQL 2012, but the problem does exist in 2005, 2008 and 2008 R2.  If I discover any further useful information, I will update this blog post, but at a high level I think that I have covered the problem and solution.

Developing Microsoft SSIS packages programmatically with EZAPI

I recently started working on a task that required me to dig into creating SQL Server Integration Services packages programmatically.  There is not a lot of info out there on this, so thought it might be beneficial for me to blog a bit as I go in case other benefit from this.

Relatively early on in my investigation I found that the internal SSIS test team at Microsoft has created and released a second level API for SSIS called EzAPI that sits on top of the native SSIS API.  Using EzAPI, you can create packages and package objects in only a couple lines of simple code, vs. trying to decipher the somewhat esoteric COM wrapper APIs for SSIS.

You can read about EzAPI on the SSIS Team Blog here.

An update was released in late 2010, which you can read about here.  This update added a lot of additional components to the API.

You can download it here (including source code and samples).

The objects that are supported are as follows:

Tasks and Containers
  • For Loop container
  • DataFlow Task
  • Execute Package Task
  • ActiveX Script Task
DataFlow Components
  • OLEDB Source
  • OLEDB Destination
  • FlatFile Source
  • FlatFile Destination
  • ADO.Net Source
  • ADO.Net Destination
  • Multicast Transform
  • Derived Column Transform
  • Sort Transform
  • OLEDB command Transform
  • Lookup Transform
  • Cache Transform
  • Data Convert Transform
  • Aggregate Transform
Connection Managers
  • OLEDB Connection manager (including specific versions for SQL Server, Oracle, DB2)
  • FILE connection manager
  • FLATFILE connection manager
  • CACHE connection manager
  • ADO.NET connection manager

ADDED IN DECEMBER 2010 UPDATE

  • Sequence container
  • For each loop container
  • Transfer database task
  • Execute process task
  • File system task
  • Execute SQL task
  • Excel source adapter
  • Excel destination adapter
  • Union all transform
  • Merge & Merge join transform
  • Conditional split transform
  • SCD transform