Silverlight diagramming, WCF, the Entity Framework and MySQL.

It’s been a while since I jotted something down on this blog…it’s been hectic for months now and the pace doesn’t seem to slow down but I think I have a few interesting things to mention.

Since the end of last year G2 has grown bigger, stronger, more popular and various big customization projects are on our workbenches. At the same time, Graphite continues to be our bestseller but most customers use it ‘as is’ without much alterations. There are however a few lines of evolution. On the one hand, Silverlight becomes more and more popular and seems to have a better market penetration than WPF. This means that also Silverlight diagramming solutions are increasingly asked for. In this context, Graphite allows an easy path to data visualization but misses the flexibility and breadth of a true diagramming framework. Equally well (and enforced by the Silverlight wave), more and more customers wish to have a dual solution; all the advantages of diagraming in WPF and the possibility to bring as much as possible on the web (Sharepoint integration, read-only viewers and so on). These evolutions and the obvious gap in our products related to this meant a necessary investment in Silverlight and a common diagramming core. Now, hold your horses and don’t expect the sky since Silverlight still remains a subset of the .Net framework and even though Silverlight v3 broadens the horizon I don’t think any day soon you’ll have a full-compatibility tale (except through XBAP, but that’s another story). In any case, here is the good-news show and some notes from the field where diagramming meets Silverlight, the Entity Framework, MySQL and WCF.

Before describing the juicy technical details let me give you an overview of the application (screenshot above) we’ve been working on recently. It’s a Silverlight application which allows one to visualize any kind of relations between different types of entities; persons, documents, sites, etc. Any kind of relation means that it allows multiple links and generic graphs but can also handle tree-like data and multiple diagram layouts. The database uses just a few tables to store the graph structure and the actual data (persons, documents, metadata) can reside in the database but could also be stored in another database, in XML files, you name it. The application also allows one to edit relations and to edit the entities; you can draw links and it will be saved in the database, you can change or add entities and so on. Finally (and this is what this blog items is mostly about) this Silverlight application can also be used as a WinForm application and accesses data through a local self-hosted WCF service. The WCF service is a RESTful ADO.Net data service which can also handle standard HTTP requests and thus acts as a local webserver. Minor features of this application is the integrated security system which can be connected to the ASP.Net membership mechanism, to ActiveDirectory services or any legacy single-sign on system you have in-house. Let me emphasize that this application boasts a full-fledged diagramming framework for Silverlight. which in many ways extends/surpasses our current Graphite product. For example, just for fun I developed in less than a day a reasonable mindmapping application for Silverlight (screenshot below).

Entity Framework

First off, I have been happily surprised by the Entity Framework (EF). I know, it’s got a long list of shortcomings and I’ve walked the whole avenue of POCO adapters, detached entity tricks, constrained Visual Studio designer and more. However, in the context of Silverlight I was pleased to experience the ease with which it worked on top of MySQL and through the ADO.Net data services (aka Astoria). Initially I wrote the whole framework for SQL Server and had later on to move to MySQL…oh miraculus! Only minor changes were necessary on the database level (no Guid’s in MySQL, no triggers e.g.) and one needs at least MySQL v5 to make it work. Note that this happy message has more its origins in the hard work which went into developing the .Net driver for MySQL rather than how open Microsoft is towards other relational database systems. In fact, I can also claim the same for other systems: EF works well with SQLite and with SQL Server Compact v3.5. There are pro’s and con’s with each and you need to tweak your model because of datatype differences, missing features (views, stored procedures etc.) but I expected a complete no-go orginally.

ADO.Net Data Services

Concerning ADO.Net data services. It works flawless with the EF, whatever the underlying database-dependent model is. In fact, if the data type signature is identical in different database systems, you only need to change one word in your code to switch from one to the other:

1
2
3
4
5
6

[System.ServiceModel.ServiceBehavior(IncludeExceptionDetailInFaults = true)]
public class CrystalMsSqlService : DataService<crystalmysqlcontext>
{
...
}
</crystalmysqlcontext>

This code is the generator of a REST WCF service on top of a EF model. The CrystalMySQLContext is the data context generated by the EF. If you wish to switch to, let’s say, SQL Server you only need to replace the data context specification and you’ll be safe for the rest of the journey.
Note, as a side note, the ServiceBehavior attribute which propagates exceptions to the surface and which is not added by default in Visual Studio. You will also want to add a try-catch in the code to convert the exceptions to a DataServiceException like this:

1
2
3
4
5
6
7
8
9
10
11
12

public static void InitializeService(IDataServiceConfiguration config)
{
    try
    {
        ..
        config.SetEntitySetAccessRule("SomeTable", EntitySetRights.All);
    }
    catch (Exception e)
    {
        throw new DataServiceException(e.Message);
    }
}

It will make debugging much easier, trust me.

Why ADO.Net data service (ADS)? You need to know that Silverlight only likes asynchronous data exchange with the outside world. Now, in order to expose data via WCF you can go via a artisanal, hand-crafted WCF service but ADS makes things easy and turns an EF into a RESTful service in a snap. In effect, it means you have all your tables and views available in Silverlight as if they were on the client. The price for this is the asynchronous behavior, which at times can be frustrating; you cannot have a data request and a data related action in the same C# method. A wisdom which I can give to you here is: don’t try to launch inner join requests from Silverlight to the REST, it doesn’t work. If you wish to do this you need to create a view in you database and make this view accessible in ADS as if it was a table. This simple fact makes SQL Server Compact immediately a bad choice because it doesn’t support views (SQL Express will do however).

WCF (ah, so much beauty!)

One of the things which kept me busy for a long time was how to demonstrate/ship a Silverlight demo without having to ship a database and without having to buy an expensive SQL Server license? Imagine you have a beautiful Silverlight application for a customer but you don’t want a whole setup involving a SQL Server database deployment (logins, connection strings etc.) and you also don’t want a complicated IIS setup wherein the Silverlight, WCF, XAP…reside. What to do? At first I tried to develop a local WCF proxy which would go to a file based data system. This meant a self-hosted WCF service, some sort of database on the client and a separate local webserver. This worked well up to the point that Silverlight kept on complaining about security issues. No mystery here: Silverlight doesn’t want to access data sources if they are not coming from the precise (includes the port number!) same origin as the xap (i.e. the Silverlight application).

Easy enough, I thought, but read on. You only need to supply the client with the clientaccesspolicy.xml or the server with the crossdomain.xml. Extend the WCF service like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40

[ServiceContract]
public interface IWebServer
{
    [OperationContract, WebGet(UriTemplate = "/clientaccesspolicy.xml")]
    Stream GetSilverlightPolicy();
    [OperationContract, WebGet(UriTemplate = "/crossdomain.xml")]
    Stream GetFlashPolicy();
...}
public class CrystalMsSqlService : DataService<crystalmysqlcontext>, IWebServer
{...
public Stream GetSilverlightPolicy()
{
    const string result = @"< ?xml version=""1.0"" encoding=""utf-8""?>
                                <access -policy>
                                    <cross -domain-access>
                                        <policy>
                                            <allow -from http-request-headers=""*"">
                                                <domain uri=""*""/>
                                            </allow>
                                            <grant -to>
                                                <resource path=""/"" include-subpaths=""true""/>
                                            </grant>
                                        </policy>
                                    </cross>
                                </access>";
    return StringToStream(result);
}
 
public Stream GetFlashPolicy()
{
    const string result = @"< ?xml version=""1.0""?>
                                < !DOCTYPE cross-domain-policy SYSTEM ""http://www.macromedia.com/xml/dtds/cross-domain-policy.dtd"">
                                <cross -domain-policy>
                                    <allow -access-from domain=""*"" />
                                </cross>";
    return StringToStream(result);
}
...
}
</crystalmysqlcontext>

It doesn’t work. Or at least, you can check that the WCF service does supply the policies but for some reasons Silverlight in my case still didn’t want to query the WCF service.

Now, because REST allows a client to query a WCF as if it’s a website and because one can serve both ascii and binary data through WCF/REST the obvious next step was to extend the WCF service with methods which would mimic a webserver behavior. Easy to do in fact by extending the service with more methods and templating the URL requests:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52

 
[ServiceContract]
public interface IWebServer
{
[OperationContract, WebGet(UriTemplate = "/clientaccesspolicy.xml")]
Stream GetSilverlightPolicy();
[OperationContract, WebGet(UriTemplate = "/crossdomain.xml")]
Stream GetFlashPolicy();
[OperationContract, WebGet(UriTemplate = "/app.htm")]
Stream GetApplicationPage();
 
[OperationContract, WebGet(UriTemplate = "/{pagename}")]
Stream GetSitePage(string pagename);
[OperationContract, WebGet(UriTemplate = "/Images/{imagename}")]
Stream GetImage(string imagename);
[OperationContract, WebGet(UriTemplate = "/Crystalizer.One.xap")]
Stream GetCrytal();
 
}
public class CrystalMsSqlService : DataService<crystalmysqlcontext>, IWebServer
{...
public Stream GetSitePage(string pagename)
{
    try
    {
        WebOperationContext.Current.OutgoingResponse.ContentType = "text/html";
        string fileContents;
        string path;
        path = string.Format(@"WebRoot\{0}", pagename);
 
        using (StreamReader sr = new StreamReader(path))
        {
            fileContents = sr.ReadToEnd();
        }
        return new MemoryStream(Encoding.UTF8.GetBytes(fileContents));
    }
    catch (Exception)
    {
        return null;
    }
}
 
public Stream GetImage(string imagename)
{
    byte[] fileContents;
    fileContents = File.ReadAllBytes(string.Format(@"WebRoot\Images\{0}", imagename));
 
    // byte[] resp = Encoding.ASCII.GetBytes(string.Format(HttpHeader, "image/png", fileContents.Length, fileContents));
    return new MemoryStream(fileContents);
}
...}
</crystalmysqlcontext>

This now turns your RESTful ADS service into a webserver which serves both the data, the site pages and site images.
One last trick is the fact that because now your WCF service actually implements two interfaces the default endpoint configuration of ADS will not work. To solve this in the self-hosted scenario you can add another endpoint as follows:

1
2
3
4
5
6
7
8
9
10
11
12
13
14

private static void HostWithWebServiceHost(Uri baseAddress)
{
 
    WCFHost = new WebServiceHost(typeof(CrystalMsSqlService), baseAddress);
    WebHttpBinding binding = new WebHttpBinding();
    WCFHost.AddServiceEndpoint(typeof(System.Data.Services.IRequestHandler), binding, "dataservice");
 
 
    WCFHost.AddServiceEndpoint(typeof(IWebServer), binding, "").Behaviors.Add(new WebHttpBehavior());
    ServiceMetadataBehavior smb = new ServiceMetadataBehavior {HttpGetEnabled = true};
    WCFHost.Description.Behaviors.Add(smb);
 
    WCFHost.Open();
}

At this point the Silverlight application will run at the client without any server or internet connection.. You can go a step further and even host the Silverlight application in a Windows form which at the same time hosts the WCF service, but that’s really a no-brainer. You simply add a webbrowser control in a WinForm and call the HTML page which contains the Silverlight control and you’re done. Note that by doing this you have a Silverlight 2 application which runs as if it’s a regular WinForm application. Still, because the WCF service serves ANY client you can also access the Silverlight application through a standard browser. Actually, this fact helps to debug the Silverlight code by attaching the debugger to the browser process. I have not found any other way to debug Silverlight if it’s not part of a Visual Studio web application (as in my case).

In my final ‘show it to the customer without deployment’ solution, I removed the database from the client setup and kept the MySQL on a publicly available location. The only things you need to watch in this case is the fact that by default EF will store the database connection credentials in the web.config. You need to create a EF compiled connection with the EntityConnectionStringBuilder and override the CreateDataSource method of the ADS service:

1
2
3
4
5
6
7
8
9
10
11
12

protected override CrystalMySQLContext CreateDataSource()
{
    EntityConnectionStringBuilder entityBuilder = new EntityConnectionStringBuilder();
    entityBuilder.Metadata = @"res://*/MySql.CrystalMySQLModel.csdl|res://*/MySql.CrystalMySQLModel.ssdl|res://*/MySql.CrystalMySQLModel.msl";
    entityBuilder.Provider = "MySql.Data.MySqlClient";
    entityBuilder.ProviderConnectionString = "server=xxx;user id=xxx;password='xxx';persist security info=True;database=illumineo";
 
    EntityConnection con = new EntityConnection(entityBuilder.ToString());
 
    CrystalMySQLContext ctx = new CrystalMySQLContext(con);
    return ctx;
}

MySQL

During the testing and deployment phase of this project I encountered various difficulties which were all related to the fact that all the assemblies you assume are not necessarily on a blank machine. Adding these assemblies to the bin directory doesn’t solve these issues, the great trick here is a configuration you need to add in order to tell the CLR these assemblies are available. During the installation of a database driver the machine.config is altered but this is not the case if you deploy a self-hosted service and the binaries are not in the GAC:

1
2
3
4
5
6
7
8
9

< ?xml version="1.0" encoding="utf-8" ?>
<configuration>
    <dbproviderfactories>
      <clear />
      <add name="SqlClient Data Provider" invariant="System.Data.SqlClient" description=".Net Framework Data Provider for SqlServer" type="System.Data.SqlClient.SqlClientFactory, System.Data, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089" />
      <add name="MySQL Data Provider" invariant="MySql.Data.MySqlClient" description=".Net Framework Data Provider for MySQL" type="MySql.Data.MySqlClient.MySqlClientFactory, MySql.Data, Version=6.0.2.0, Culture=neutral, PublicKeyToken=c5687fc88969c44d" />
    </dbproviderfactories>
 
</configuration>

Silverlight only likes it async

Concerning the asynchronous behavior of Silverlight I can only say: you need a different mindset to live with it. In my Silverlight application I invested quite some time to create an MVC pattern on the basis of asynchronous data access. Because Silverlight handles templating/styling differently and doesn’t have a databinding mechanism which support element names all of this was tricky at times but in the end I think I was able to cook a very clean and felxible Silverlight framework which in fact has not much affinity with diagramming and could be used as a generic framework (just like our O2 framework for WPF).

Commercial rands

I mentioned in the introduction that one of the aims of this project was to have a common core between WPF and Silverlight. Bad news here: you cannot share assemblies between Silverlight and WPF, you cannot share template, you cannot share databinding patterns, you cannot…well, lots of things. You can find on the net different ways to handle these difficulties (pragma’s in your code for instance) but I haven’t found any of these satisfactory. So, the end result in this context is a big deception, until Microsoft brings on stage a clear strategy targetting this scenario you have to live with duplicated code. For our diagramming aims this means we have a completely different Silverlight library which covers a lot of the features available in G2 but is nowhere near identical in code as G2. On the surface (read: our commercial offering) we now have

• an enterprise level diagramming framework (product: G2) for WPF and a MEF-based framework for WPF applications (product: O2).
• a very complete diagramming framework (product: L2) for Silverlight and a MVC-based framework for Silverlight applications

which in look-and-feel will give you pretty much the same experience. A BPM modeller or a workflow application in Silverlight with the same functionality in WPF…it’s all possible but not on the basis of a common core. In commercial terms this means that a customer wishing to have a dual Silverlight/WPF product needs to pay twice for customizations but can share a common data access layer hidden behind a WCF facade. Speaking about commercial moves I should also mention that The Orbifold has acquired recently two new products which were previously in the open source realm: Denis’ property grid for Silverlight and WPF, as well a Petro’s S# scripting framework (aka Script.Net). More about this in the near future.

There is so much more to say about our other projects and our latest WPF magic but I’ll stop here and leave if for another blog post.