Head over to this page to see my latest mashup (aka data visualization experiment) with d3.js and KendoUI. A lot of fun really to mix ‘nd match animations, effects and visual elements into something which should represent an HMI dashboard (human-machine interface) or a dashboard of real-time data sources. No efforts was made to represent anything in particular since my focus was on trying out something on an artistic level rather than on a business level. Still, now that I have a good understanding of the technicalities involved I estimate it no more (or less) difficult than Silverlight or WPF to develop a true business dashboard (Sharepoint, WebSphere or whatever your local flavor is).

Much appreciation goes to the guys behind KendoUI and Mike Bostock’s amazing d3 library. Sheer pleasure to develop with these libs.

Official for Q2

As you already know it is currently in beta state (even we think it is more than a beta) but with the upcoming Q2.2012 release the RadDiagram will become official control. That is not the only major thing that is coming; we are already working on lots of new cool features and examples. Let’s just name a few…

New Features

• Shapes
  • Hierarchical groups - you’ll be able to group shapes together and manipulate them accordingly.
  • Custom connectors - specify exact point on the shape where a connection could be pinned.
• Connections
  • Bezier curves – read more about Bezier here.
  • Polyline improvements
  • Routing improvements
• Diagram export
  • Printing
  • Export to Image
• Diagram Canvas improvements
  • Rulers - we’ll add vertical and horizontal rulers for easier positioning of the shapes.
  • Thumbnail (mini-map) view – this is a very cool feature. The control will visualize the whole diagram “world” and what part of it you’re seeing.
  • Auto-layout and auto-aligning
  • Improved handling of dropped content
• Tree layout and variations – this is something huge. You’ll be able to easily choose between different types of tree layouts like mindmap, tip-over-tree or radial trees and forests.
• New Extension assembly – It will contain the Toolbox control and some other cool stuff to help you create easily diagramming applications. It will be prety much like the current Features assembly that is part of the demos distribution.

New Samples

• Mindmap example – we are creating a mindmap example that will demonstrate the power of the RadDiagram framework.

Connection Routing

Connection routing is an algorithm or a set of algorithms that should “trace” the route of link/connection, that takes into account the other nodes/shapes and some other heuristics.

Tree Layouts

Have a look at few screenshots of the RadDiagram showing the new tree layouts.

That is our current plan, but stay tuned for more information on the RadDiagram. Your feedback and ideas are more than welcomed.

» Full source code of the demo application here. You can download a trial or purchase RadDiagram here.

The new RadDiagram framework brings, together with a host of new exciting things in Telerik’s 2012 Q1 release, a new world of possibilities to the WPF and Silverlight community. This article attempts to show you that Telerik’s RadDiagram is easy-to-use and that it differs from other Telerik controls in that it more closely represents a framework than a control. It’s an invitation to augment and customize the RadDiagram toolset in function of your application and business context. That said, we have to tell you straight away that RadDiagram, as released with Q1, is just the beginning. We have at this point put down the foundations for lots of exciting things to come and you can expect to see this RadDiagram branch off in various directions in coming releases. Great care and much energy were put in making it a flexible, robust, and broad framework for future developments. We have tried to make your (and our) task of customizing things easy while retaining the possibility to make the complex possible. 

The dashboard designer described in this article aims to highlight various development tasks related to RadDiagram and how to integrate it into a real-world LOB application. It also tries to underline the fact that the diagramming world is a very broad subject on many levels. In its most basic form a diagram (graph, network) is just a set of shapes and lines connecting these shapes, but diagramming frameworks are also used for a wild spectrum of other tasks:

• For so-called  HMI (human-machine interface) diagrams where the shapes contain data from sensors and other real-time data providers
• In geographic maps to position labels (using appropriate layout algorithms) and to calculate shortest paths or other extremum, graph-based things
• To design workflows in which case shapes and connections are tightly bound to an underlying API
• To simulate processes and represent data logic (so-called system dynamics) in which case the graph effectively represents a flow of data and logical switches. The typical example being the diagrams used in rendering applications where texture and material properties are modeled through graphs.
• For data visualizations and data analysis; social networks and IP networks, for example

[Picture above] Typical dashboard in our sample application representing an overview of stock-related data coming from Yahoo Finance.

Yet another context wherein diagrams come in handy is when you need a flexible surface on which you need to drop elements to design a document or data dashboard. While graphs are intrinsically related to links and connections, nothing prevents you from using a diagram framework without them and thus reduce it to a layout or design engine (a form designer anyone?). Our financial dashboard designer uses the RadDiagram surface in both ways; to design data dashboards (no connections here) and to represent the hierarchical structure of these dashboards (relationship graph).

How does the application gets its relational structure? Put simply, the hyperlinks you add on a dashboard can be internal; you can create a link which jumps to another dashboard page much like hyperlinks in a web site.

The important bit here is the fact that across the application there is only one RadDiagram instance used in multiple ways and that all the dynamics occur through the (de)serialization. In fact, the generic flow in the application is as follows:

• At any moment there is only one active page which corresponds to a diagram. You can remove or add pages as you please, but only one is active at a time, much like a TabControl tab-page in fact.
• From the toolbox data elements can be drag-dropped onto the diagram (dashboard) and real-time data is fetched from the Yahoo Finance service (you need to specify a stock symbol, of course).
• The elements can be customized, if needed, by double-clicking them.
• When a page is saved it’s being serialized to the isolated storage space and the hyperlinks are analyzed thus creating a graph (an hypergraph in fact) of relations between pages/dashboards.
• Whenever a page is loaded from the storage the corresponding diagram XML is deserialized.

This flow demonstrates the following aspects of RadDiagram:

• The flexible serialization pipeline you can plug into to include your application-specific data.
• The ease with which you can customize the RadDiagram toolbox to incorporate your own bits.
• The way you can use the many switches and properties of the diagramming surface to model a particular behavior, enabling/disabling e.g. connections, resizing, rotation, and so on.
• The speedy loading and unloading of diagrams, which effectively enables a multi-view paradigm, bypassing the need for a TabControl
• The way you can use diagramming shapes as containers for other Telerik controls, for your own data or to act as recipients for real-time data sources

Now that you have an overview of what the demo application does and its various features let’s dig a bit more into the technical details.

A note on the Yahoo Finance API

The dashboard designer uses realtime financial information from the Yahoo Finance API:

• You can drag-drop the ‘company profile’ shape onto the diagram surface and it will fetch the company profile for the stock symbol of your choice.
• General financial information, timeline graph and other data can be fetched in the same way. Each type of information is presented in a different shape (format) and demonstrates how you can combine Telerik’s controls into the RadDiagram framework
• RSS news feeds can be appended

The code to access the Yahoo Finance API is quite straightforward and one can, of course, reuse these controls and the underlying code for other data sources.

Creating custom shapes with RadDiagram

The RadDiagram framework comes with a standard set of flowcharting shapes (geometry shapes) but this shouldn’t prevent you from enlarging it with your own set. In fact, diagramming is such a vast domain and its integration within the business context is so tight that any diagramming framework would fail to deliver a complete set of shapes. Rather than trying to offer lots of shapes RadDiagram offers you a flexible and straightforward route to designing custom shapes and embedding (Telerik or third-party) controls.

The generic recipe for creating a custom shape goes as follows:

• Inherit from a base class, either RadDiagramShapeBase or RadDiagramShape. The latter one contains a Geometry property, if you need to have some geometry inside your shape.
• Design a control or content template in XAML. This is the point where you can go wild in combining the Telerik controls and all the goodness of the XAML framework.
• Optional: Add some (dependency) properties reflecting the business context. For example, in the custom shapes provided with the sample you’ll often see the stock Symbol property defined.
• Optional: Override a couple of serialization methods, if the custom properties need to be saved/loaded/copied.

Let’s take, as an example, the GaugeShape from the sample, which (see adjacent picture) displays the latest price and price change of a stock. The shape inherits from RadDiagramShape

1

public sealed class GaugeShape : RadDiagramShape

and defines a custom Symbol dependency property

1

public static readonly DependencyProperty SymbolProperty = DependencyProperty.Register("Symbol", typeof(string), typeof(GaugeShape), new FrameworkPropertyMetadata(null, OnSymbolChanged));

Whether it’s a standard or a dependency property depends really on what you want to do with it, see
the MSDN documentation on this.

Next, you need to define the visual appearance of the control. Here is an important point you need to consider; do you need the adorning elements (resizing adorner, connectors, rotation handle) of the RadDiagram framework to appear? If so, the easiest way is to base your XAML style on the base style (using the BasedOn property) from the framework;

1

<Style TargetType="{x:Type local:GaugeShape}" BasedOn="{StaticResource {x:Type telerik:RadDiagramShape}}">

As a result, all the adorning diagramming elements are already present and your custom shape will take part in the internal mechanics of RadDiagram. If you omit the directive, then the shape will still be part of the diagram, but you will not see or be able to use the resizer, rotation handle, and so on.

From this point on you can define inside the style all you wish, but there are some gotchas:

• The ContentTemplate does not immediately give access to the (dependency) properties of the class, but instead exposeses the Content property. So, if you want to access the DataContext inside the Content template you have to either assign this data context to the Content, or you need to walk up the visual tree a bit (see below).
• The EditTemplate can be (and usually is) a totally different template than the ContentTemplate; it allows you to define how the control looks when the user double-clicks the shape and shifts to ‘edit’ mode. A good pattern here would be to set some data in the DataContext, assign the DataContext to the Content, and to access this data bucket both from the ContentTemplate and the EditTemplate. In situations where the EditTemplate is not sufficient you can hook up the routed DoubleClick event and redirect the default editing behavior (showing the EditTemplate) to some custom behavior. For example, in the sample you will notice that the TextShape shows a custom window on double-clicking. One essential elements here is that you need to halt the event bubbling by setting ‘Handled = true’ otherwise you will have both a custom handler active and the EditTemplate visible.

In the content template of the GaugeShape you will see the following bits:

1
2

<TextBlock Text="{Binding Path=Name}" Foreground="{StaticResource MetroAccentBrush}" FontWeight="Bold" HorizontalAlignment="Center" />
<telerik:RadRadialGauge OuterBorderBrush="#FF08FF00" Grid.Row="1" Background="WhiteSmoke">

The TextBlock accesses the Name property because the Content contains the Quote information:

1
2
3

var quote = new Quote(Symbol);
YahooEngine.Fetch(new List<Quote> { quote });
this.Content = quote;

On the other hand, if you want your shape to be stretched and resized according to the outer resizing adorners, you should set the following properties:

1
2

<Setter Property="HorizontalContentAlignment" Value="Stretch" />
<Setter Property="VerticalContentAlignment" Value="Stretch" />

Finally, in order to have the Symbol property persistent, you need to tell the framework how to serialize it. While this might sound complicated we really made it simple for you to perform this step.  The serialization of a property (say, the Symbol property) amounts to adding an entry in a dictionary:

1
2
3
4
5
6

public override SerializationInfo Serialize()
{
    var info = base.Serialize();
    info[“Symbol”] = this.Symbol ?? "";
    return info;
}

Conversely, if you want to pick up the saved value of the Symbol property you just ask the dictionary for the value:

1
2
3
4
5

public override void Deserialize(SerializationInfo info)
{
    base.Deserialize(info);
    if (info[“Symbol”] != null) this.Symbol = info[SymbolTag].ToString();
}

There is more under the hood than this straightforward serialization alone, please consult
the RadDiagram documentation for more serialization goodies.

Once all this is in place you can add the custom shape to the diagram and bind it to the application data like so:

1
2
3
4
5

var shape = new GaugeShape
    {
        Position = new Point(10, 120)
    };
Diagram.AddShape(shape);

Many more possibilities and extension points are at your disposal, several of which are demonstrated in the custom shapes inside the sample application. This elementary overview, however,should get you going and we have made the documentation as complete as possible to guide you through the ins and outs of the RadDiagram framework.

The toolbox: how to add your custom items

The RadDiagram framework does not provide a Toolbox control out-of-the-box. It is too specific a part of the story and could just be used for some diagramming applications. That said, it is not part of the core framework, but we wrote several lines of code that can be used and easily customized for your needs. There is an Assembly that we called Features (Telerik.Windows.Diagrams.Features.dll), which is an extension of the core RadDiagram framework, and contains some nice features. For example, there is a ToolBox control there. It is a very simple Hierarchical ItemsControl that you can populate with a hierarchical data source or manually with ToolBoxGroups and items. On the other hand, the items are not much more than ContentControl, so you can
effectively add any UI Elements you would like to the toolbox(i.e. you can even add regular shapes from the RadDiagram framework)

1
2
3
4
5
6
7
8
9
10

<expandertoolbox:DiagramToolbox>
<expandertoolbox:ToolboxGroup Header="Shapes">
       <expandertoolbox:ToolboxItem Header="Rectangle" Description="A simple rectangle">
             <local:ShapeToolboxItem Geometry="{telerik:CommonShape ShapeType=RoundedRectangleShape}"/>
       </expandertoolbox:ToolboxItem>
       <expandertoolbox:ToolboxItem Header="Ellipse" Description="A simple rectangle">
             <local:ShapeToolboxItem Geometry="{telerik:CommonShape ShapeType=EllipseShape}"/>
       </expandertoolbox:ToolboxItem>
</expandertoolbox:ToolboxGroup>
</expandertoolbox:DiagramToolbox>

But the interesting part here is how to handle the drop from the Toolbox to the diagram surface. To make our lives easier, we’re using the Telerik DragDropManager, which automatically handles the start-drag operation of the toolbox item. Thus we need to handle the drop operation at the other side, on the diagram surface.

1

this.diagram.Drop += this.OnDropToolboxItem;

Let’s take a look at the handler as well.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

var item = (e.Data as DataObject).GetData("Telerik.Windows.Diagrams.Features.ExpanderToolbox.ToolboxItem") as ToolboxItem;
if (item != null)
    {
        IToolboxItem toolboxItem = item.Content as IToolboxItem;
        RadDiagramShape shape = null;
        var position = e.GetPosition(this.diagram);
        if(toolboxItem != null)
        {
            shape = toolboxItem.CreateShape(position);
        }
 
        if (shape != null)
        {
            this.diagram.AddShape(shape);
            e.Handled = true;
        }
}

The trickier part is that each type of ToolboxItem implements the CreateShape factory method (from the IToolboxItem interface), which creates a new, configured shape for this type of toolbox item.

1
2
3
4
5

public override RadDiagramShape CreateShape(Point position)
{
var shape = new ImageShape { Location = this.ImageLocation, Address = this.Address, Position = position, NavigateUri = this.NavigateUri };
return shape;
}

The RadDiagram surface properties

There is one more interesting story in this application: How to switch between Edit and Design surface modes. The RadDiagram framework provides you with plenty of properties that can control the read-only state in different aspects. Mentioning these properties here affects the run-time through UI behavior and does not enforce changes through the APIs.

They are:

• IsEditable – Indicates whether editing is enabled.
• IsPanningEnabled – Indicates whether panning is enabled.
• IsZoomEnabled – Indicates whether zooming is enabled.
• IsResizingEnabled – Indicates whether shape resizing is enabled.
• IsRotationEnabled – Indicates whether shape rotation is enabled.
• IsDraggingEnabled – Indicates whether shape dragging is enabled.
• IsConnectorsManipulationEnabled – Indicates whether connection creating is enabled through the connectors.
• AllowDelete – Indicates whether item deletion is enabled.
• AllowDrop – Indicates whether the diagram can accept drop.
• AllowCopy – Indicates whether item copying is enabled.
• AllowPaste – Indicates whether item pasting is enabled.
• SelectionMode – Can choose the selection mode through enumeration:
  • None – selection is disabled.
  • Single – only one can be selection either with Rectangle selection tool or pressing CTRL key.
  • Multiple – selecting items on each click and with Rectangle selection tool.
  • Extended – selecting items with CTRL or using Rectangle selection tool.

To conclude

[Picture above] Please visit the documentation on RadDiagram for much more info on other aspects not touched by this overview article.

While this demo application shows off some of the more juicy features of RadDiagram it is by no means the end of the story. You are invited to continue reading on:

• RadDiagram for Silverlight and the many additional possibilities Silverlight offers to display rich diagrams in the browser
• The MVVM mechanism for RadDiagram
  
• The inter-operability of the clipboard between RadDiagram for Silverlight and WPF
• The shared WPF and Silverlight document format, which allows you to exchange diagrams between the two technologies.

1-2-3 example

Create a website in Visual Studio (I’ve taken a MVC one here) and right-click on your project, go to the Nuget package manager and search for ‘Processing.js’ in the online section

Click on ‘Install’ and a couple of JS libraries will be added to your Scripts folder.

Either in your aspx/html or in the ‘_layout.cshtml’ file add a reference to the ‘Processing.js’ library. You can simply drag-drop the file from the Solution Browser into the text editor and Visual Studio will automatically create the correct reference. From this point on the possibilities are endless, have a look at the Processing ehibition for instance.

For example, if you wish to add a clock to your page change the ‘Index.cshtml’ to the following:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53

@{
    ViewBag.Title = "Home Page";
}
<h2>
    A simple processing.js clock
</h2>
<p>
    <canvas id="canvas1" width="200" height="200">
    </canvas></p>
<script id="script1" type="text/javascript">
 
    // Simple way to attach js code to the canvas is by using a function
    function sketchProc(processing) {
        // Override draw function, by default it will be called 60 times per second
        processing.draw = function () {
            // determine center and max clock arm length
            var centerX = processing.width / 2, centerY = processing.height / 2;
            var maxArmLength = Math.min(centerX, centerY);
 
            function drawArm(position, lengthScale, weight) {
                processing.strokeWeight(weight);
                processing.line(centerX, centerY,
        centerX + Math.sin(position * 2 * Math.PI) * lengthScale * maxArmLength,
        centerY - Math.cos(position * 2 * Math.PI) * lengthScale * maxArmLength);
            }
 
            // erase background
            processing.background(224);
 
            var now = new Date();
 
            // Moving hours arm by small increments
            var hoursPosition = (now.getHours() % 12 + now.getMinutes() / 60) / 12;
            drawArm(hoursPosition, 0.5, 5);
 
            // Moving minutes arm by small increments
            var minutesPosition = (now.getMinutes() + now.getSeconds() / 60) / 60;
            drawArm(minutesPosition, 0.80, 3);
 
            // Moving hour arm by second increments
            var secondsPosition = now.getSeconds() / 60;
            drawArm(secondsPosition, 0.90, 1);
        };
 
    }
 
    var canvas = document.getElementById("canvas1");
    // attaching the sketchProc function to the canvas
    var p = new Processing(canvas, sketchProc);
    // p.exit(); to detach it
</script>
<div style="height: 0px; width: 0px; overflow: hidden;">
</div>

Run/display this page and you’ll get something like the following:

The Value of Data Visualization from Column Five on Vimeo.

Finding the needle in the haystack is now more pressing than before. The more data we produce and have access to, the more the need for something intelligent to get the information out of the data becomes important. Obviously the ever-increasing amount of data and the shift to global data stores like the cloud have increased the need for better visualizations techniques and innovative thinking.But the amount of noise has also increased and the need to choose through it. How to filter out of the billions of tweets the stuff that matters to you? The only way to do that is by choosing; what matters to you is related to the things that you choose for and which have a direct relevance in your life (personal or professional). Before the filtering comes the criteria and the criteria are dictated by the business or personal context combined with a more-or-less precise definition of what the solution or answer should be like. Usually a question has many answers or complementary answers, much like a mathematical problem has a one or more solutions depending on the target space within which the solution should be found.

Although I find it refreshing to see that data visualization and presentation techniques are now more important than ever before, it’s not for myself the reason why this field is interesting or fertile. Data visualization is useful as a product for the end-user or consumer but is equally well interesting for the producer or creative inventor because it combines various fields. Like nothing else you need to be partly an artist, partly a scientist to come up with something beautiful yet meaningful and interesting. You need to understand sometimes deep technical details and exotic programming concepts while having a good understanding of the business context and what a customer needs. You need to fathom the (business) question and at the same time have a vision, which usually means a mixture of intuition and expertise. You need to understand the language of different people and sub-cultures, read through minor details, yet see through the details and go beyond to deliver a broad concept. Having a bird’s-eye view while knowing the murky details on a low-level demands a mind which can stretch across all levels of thinking. Can you make the simple complex and the complex simple? Can you see color where others see mathematical formula’s? Can you see sound where others see data and shapes? Can you see code where someone sees motion? Can you see patterns where others see noise? Can you see information where others see data?

• Visualizing.org
• Information is beautiful
• Data Visualization Showcase

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

type Node =
    int (* node identifier *) *
    'TDaton (* node data *)
 
type Connection<'TDaton> =
    int (* source Id *) *
    int (* sink Id *) *
    int (* weight *) *
    'TDaton (* connection data *)
 
type Adjacency<'TDaton> =
    Connection<'TDaton> list (* a list of connections to other nodes*)
 
type Atom<'TNodeDaton, 'TConnectionDaton> =
    Node<'TNodeDaton> * Adjacency<'TConnectionDaton>
 
type Graph<'TNodeDaton, 'TConnectionDaton> =
    Atom<'TNodeDaton, 'TConnectionDaton> list

The connections are directed links between nodes and the weight tuple factor is taken out of the generic data bucket in order to make weighted graph analysis possible. One could replace the generic data attached to the connection by an interface which would require a weight property, but here again this would complicate things in code. In any case, in the documentation of the library I have added an example of how to attach polymorphic data buckets to nodes and connections.

The adjacency type is just an alias for a connection list and, finally, the graph is as advertised above just a list of nodes with their adjacency list. The Atom type is a type alias for one single graph (list) element and comes handy in the algorithms.

From these few type definitions you can spend the rest of your life doing graph analysis and implementing all the known (or unknown) graph algorithms; shortest path, Hamiltionian cycles, connected components and so on. At this point the current implementation contains the most used ones and the emphasis was also on having a quick (and easy) way to display the graph (and their properties).

Obviously, if you want to implement some graph algorithms you need all the basics like; adding nodes, removing connections, setting node data, fetching upstream and downstream nodes, testing whether a link exists and so on. In addition, you also need a way to easily create graphs or generate random graphs (with a fixed node degree for example). Since most of the graph analysis focuses on paths and structure it’s often not necessary to adorn the graph with data or weights, hence the possibility to define graphs through a more simple structure:

1
2
3
4
5
6
7
8
9
10

 
let pregraph =
        [
        (1, [2]);
        (2, [3;5]);
        (3, [4]);
        (5, [4]);
        ]
let graph = Orbifold.FGraph.ParseList pregraph
graph |> GraphVisualizer.Show

This will launch the visualizer and produce the following simple graph:

The power of F# resides in the handling of lists and avoiding loops as one would in C# tremendously helps in situations like this. One should compare the ParseList method implementation with what it would take in C#:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

///<summary>Parses a simple list made out of tuples of (int, int list). The first integer is the index and the list is the list of nodes indices to which the current node is connected.</summary>
let ParseList (list : (int* int list) list) : Graph<Object,Object>=
        let g = ref Graph.Empty
        let linkedIds: int list ref = ref []
        let containsNumber number list = List.exists (fun elem -> elem = number) list
        g.Value <- (list |> List.map (fun (idx,adjs) ->
                                let newNode : Node<Object> = (idx, null)
                                let ad = List.map (fun t->
                                                        if not (containsNumber t !linkedIds) then linkedIds.Value<- t::!linkedIds
                                                        let con : Connection<Object> = (idx,t,0,null)
                                                        con
                                                   ) adjs
                                let atom : Atom<Object,Object> = (newNode,ad)
                                atom
                          ))
        (*we auto-add the node if it appears in the connections*)
        linkedIds.Value |> List.iter (fun t ->
                                             if not (NodeExists t !g) then
                                                            let n : Node<Object> = (t,null)
                                                            let c : Adjacency<Object> = []
                                                            g.Value<-(n,c)::!g
                                      )
        g.Value

The interesting part of the library sits in the analysis module where one encounters the unavoidable traversals and where I tried to also implement useful variants like the depth-first traversal of edges:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

let DFTEdgeVisit (g:Graph<_,_>) (start:int) (processEdge: int*int->unit) =
        let nextToVisit = Stack<int>()
        let discovered = HashSet<int>()
        let v = ref 0 (* current vertex *)
        ignore(nextToVisit.Push(start))
        ignore(discovered.Add(start))
        let parents = Dictionary<int,int>()
        while nextToVisit.Count > 0 do
            v.Value <- nextToVisit.Pop()
            if parents.ContainsKey(!v) then processEdge (parents.[!v],!v)
            let someDescendants = GetDescendants !v g
            match someDescendants with
            |None -> ()
            |Some(desc)-> List.iter (fun d -> if not (discovered.Contains(fst d)) then
                                                                                    ignore(parents.Add(fst d,!v))
                                                                                    ignore(discovered.Add(fst d))
                                                                                    ignore(nextToVisit.Push(fst d))) (List.rev desc)

Here again, one would use in C# Action<> parameters for the visitors but in F# things are made easy since functions and variables are on the same footing.

Some of the algorithms, like the shortest path between two nodes, are quite straightforward in C# but are really an headache when trying to convert to purely functional analogs. At this points the library does its best to honour the functional paradigm (also because often it represents immense performance gains w.r.t. the more linear non-functional versions) but sometimes the traditional algorithms are more readable and hence more accessible for developers who wish to alter the code.

Finally, I’d like to highlight the graph visualization part which resides in a separate library. When experimenting with F# and graphs in the F# interactive mode you can easily display the graph by feeding it to the GraphVisualizer.Show method:

1
2
3
4
5
6
7
8
9

let g =
        [
        (1, [2]);
        (2, [3;5]);
        (3, [4]);
        (5, [4]);
        ]
g |> GraphVisualizer.Show;;
GraphVisualizer.ShowShortestPath(1,4,g) ;;

You can also use the graph visualizer as a WPF component in your application. The control is a classic MVVM control with an ItemsSource accepting a Graph structure. Typically you’d use it as follows:

1

    <Visualizer:GraphControl x:Name="Diagram" Grid.Row="1" Tension="{Binding Value, ElementName=TensionSlider}" ItemsSource="{Binding Graph}"/>

where the tension determines the tension between the node in the spring-embedder layout algorithm. I’ll add some more layout algorithms in the next pass (tree layout variations in particular).

The WPF component also has easy access to the analysis like for example the shortes path:

1
2
3
4
5
6
7
8
9
10
11
12
13

let g2 = Graph.ParseList [
                            (1, [3;6;9]);
                            (2, [4]);
                            (3, [2;5]);
                            (5, [20]);
                            (6, [7;8]);
                            (7, [12]);
                            (9, [10;11]);
                            (12, [13]);
                            (20, [12]);
 
                        ]
this.Graph <- Graph.Analysis.ColorizeShortestPath( 1,13, g2)

where “this.Graph” is here the MVVM property of the model bound to the GraphControl. The result would be the picture below.

Some final remarks and questions regarding this project:

• all’s F#; the custom diagramming control, the library and the MVVM stuff. It would make sense to use this library in a C# context (to benefit from the F# performance) but I haven’t put any energy in this. Not sure it’s a priority since our other libraries (G2, Lynx) contain all this as well.
• this is work in progress and I spend a lot of time documenting things as well as creating sample code, I reckon things will settle by September this year.
• making this open source would be a good thing for the community but I’ll have to sleep over it. You can try to convince me.
• the WPF control is a templated control and can be customized. On the other hand, the arrow and node visual are hard coded. It could be useful to template them as well and in this fashion enable databinding of the databuckets defined above to the visuals. However, this library is not a full-fledged Visio replacement, rather it focuses on graph analysis. Does is make sense to invent a mechanism to enable more line-of-business scenario’s? I think our existing graph libraries are filling the need there. The only reason I would put energy in this is that at this moment there is no graph visualization library for F# on the market.
• What about Silverlight? Well, I don’t see a technical issue there but I am less sure about the audience doing Silverlight development in F#. Is there an audience for this?

Thanks to Tomas Petricek for some useful suggestions and the Flying Frog for insightful directions.

• is a computational expression the same as a workflow, is it a monad?
• how do math articles about monads map to articles about functional programming?
• are the methods defined by F# as part of a computation expression dictated by the idea of a monad? How to translate the concepts from categories to functional programming and back?
• why should I care about monads, they never appear in C# or JavaScript…!?
• how can the Haskell or OCaml stuff be translated to F#? Though Haskell has a whole lot of info regarding monads, it’s not always easy to see how it maps to the F# syntax or way of thinking.

Let me say straight away that understanding monads will not directly solve your marketing challenges, nor will it make writing software easier, not is it a magical recipe to simplify the data access layer of your website. However, it does

• make you a better programmer in the same way that learning design patterns (anti-pattern etc.) teaches you to recognize structures and solutions on a higher level of abstraction
• shed light on the relationship between domains which seem unrelated (exception handling, state management, graphs and networks, asynchronous calls, domain specific languages and so on)
• suggest you to rethink software structures and how typical programming challenges are solved
• open up a large field of ideas in the overlap of software development and maths which are fun to explore.

I don’t expect to see monadic stuff appear in Silverlight animations or in CSS style sheets but for sure you’ll see it appear in

• the NoSQL movement and graph analysis algorithms
• the financial and insurance business where F# is at this moment a big hit
• DSL developments and meta-programming

and who knows what the future brings?

A simple idea

We are surrounded by data collections and structures, sets of interrelated data and hierarchies:

• a family and its members where relationships are defined by concepts like father, mother, sister, uncle and so on
• company hierarchies with employees and relationships defined by team structures, divisions and so on
• modules of code and calling/instantiation relations where exceptions are bubbling from down the stack to the upper (UI level if not properly handled) parts
• algorithmic financial transactions (flows) based on some intricate logic

and on a more scientific level even more where one notices certain similarities and intriguing analogies:

• genes and memes: the formation of genetic structures vs. the formation of thoughts and ideas
• the spreading of viral diseases vs. the spreading of computer viruses
• the six-degrees of freedom appearing on the internet vs. the plex of actors in the IMDB database

In a lot of cases similarities are just accidental (it’s because a banana and a lemon are yellow that there is deep connection, except the shared pigment) but there are equally well amazing symmetries which can be made precise. Category theory is a theory which makes this idea of ‘being similar’ exact and at the heart of it is really a very simple notion:

• you have collection of objects
• a collection has a certain structure
• you can map objects from one collection to another and the relationship can also be mapped.

From this we take a bold step a define a (pedestrian) category as follows (more precise definition below):

Maybe it should be called at this point a pedestrian category or something because this is of course not very precise; this definition would turn pretty much everything into categories and functors! At this point I only want to help you understand where the real maths and the hard F# stuff is coming from and make it more acceptable.

A morphism is something which morphs one thing into another, something which transfers one thing into another. You will also encounter in the literature similar concepts like

• homomorphism: it emphasizes even more the idea of morphing. The prefix ‘homo’ meaning ‘the same’ and ‘morph’ means ‘changing’. Technically speaking the term homomorphism comes from group theory and morphism refers to the more general idea which can be applied outside group theory and Lie groups.
• endomorphism: ‘endo’ means ‘in itself’ and this emphasizes that a morphism is not going somewhere else but stays in the same package. So, an endomorphism of a family would map one person onto another inside the same family. A morphism would in general morph one family onto another.
• epimorphism: here the emphasis is on the fact that the morphism reaches the whole target collection.
• isomorphism: in this case the morphism is mapping elements one-to-one and onto the target. The notion is a very important one and appears pretty much everywhere in maths and science. In essence it says that if two things are isomorphic they are indistinguishable from one another and anything.everything you say about one is valid for the other.

Some more simple examples

In order to move away from the heuristics above towards some more serious definitions I need to highlight more (simple) examples.

Example

Take the real number (floats, if you prefer) with an operation called ‘addition’. Take the same real number with an operation called ‘multiplication’. We call them objects in the category of ‘real numbers with an operation’. A morphism f between the two objects is This morphism has the amazing property that: and this property is shared with many other types of collections. In addition, you should also notice that zero is mapped to one and that zero is the identity (null element) for the addition while the number one is the identity for the multiplication.

Example

Take the collection of lists of strings (List in C# and ‘string List’ in F#) and consider the morphism called ‘concatenation’ between lists. Take the collection of lists of integers on the other hand and with the same concatenation of of lists as with the string collection. A functor here is the List.map operation of F# and you should note that it doesn’t matter whether you concatenate things first and then apply the map or first apply the map and then concatenate things. In a mathematical fashion you can write that if F is the List.map action and f, g are morphisms then

Some simple list manipulations

There are some well-know examples of categories and monads like the so-called maybe monad, the identity monad and the state monad but I think the easiest way to explore or check the ideas is by means of the concat monad over the category of lists of integers, called from here on List (in bold). It consists of the following:

• sets and subsets of integers: [], [1], [1;2;3], [[1], [5;6]]…are examples of objects in this category
• the return operation which returns a singleton set from any given element
  
• the identity operator which returns whatever is given:
  
• the concatenation operator which concatenates list:
  

Now, you can play with these operations and discover some interesting properties like the fact that:

Property

Indeed if you take an arbitrary set, say [[[a];[b]];[[c;d];[[f]]], an apply the left hand side you get:

The other side of the equation tells you that:

In a similar fashion you can show that

Property

Now, let’s introduce a *-operator which acts on morphisms f in the List category:

This star-operator has also some interesting features which immediately generalize to arbitrary monads later on, namely:

Property

All these things are really easy to show inside this List category but have a deeper meaning on a general level.

Some simple maths

At this point you are ready to receive the general definition of a category and a monad.

The only difference here with the pedestrian definition above is that morphism need to satisfy the associative constraint and the existence of an identity.

Definition
A functor is a map F from one category to another which maps both objects and morphisms in such a way that

At this point you should see how this abstract definition corresponds to the various examples above, in particular the List category in the previous section. In the literature you will also see that the collection of morphisms between two objects A and B is denoted by Hom(A,B), which is an abbreviation for ‘Hom-omorphisms’.

In the previous section you was that the List category of integers has some interesting features when you combine the (List.map), the return, the id and the concat operations. The general definition of a monad is simply expressing the fact that many structures have the same set of properties. Let me first rephrase it in this specific category;

The List category has

• an operation (actually an endofunctor) (List.map) which maps lists onto lists
• the return operation creates a singleton from an element
• the concat operation together with the return operation has the following properties:
  

and this turns the triple (List.map, return, concat) into a monad.

In general, a monad is a triple which satisfies the same set of properties and you’ll see instead of concat and return the following:

Don’t be scared away by all this, it’s just an abstract reformulation of the simple concatenation and mappings of lists above! I do need however to push things a bit further because the way a monad is defined in maths is in general not the way you’ll recognize it in F# or Haskell. Usually you’ll see the equivalent formulation by Heinrich Kleisli who invented the Kleisli triple and which turned out to be an equivalent definition for monads. That is, it can be shown that there is a one-to-one correspondence between a Kleisli structure and a monad.

and you should immediately recognize the properties we mentioned above in the context of the List category. The proof that the two definitions are leading to the same structure is not difficult but it wouldn’t be useful at this point (unless you are math oriented and you need to see it to understand it).

Some simple F#

The prototypical example in the context of F# workflows is the ‘dividing by zero’ problem; one cannot divide by zero. If you have a computation where at some point an exception is raised (either explicitly or by the runtime) it doesn’t make sense to continue a computation but maybe you want to handle the exception.
The way to handle this in F# is by using a label attached to a number; if the label says ‘failure’ it doesn’t matter what the value is, if the label says ‘success’ it makes sense to read the value. It’s a way to include infinity in the constraint range of values the runtime can handle:

1

type Result = Success of float | Infinite

Next, instead of the usual division you define a divide function like so

1
2
3
4

let divide x y =
  match y with
  | 0.0 -> Infinite
  | _ -> Success(x/y)

and plug it into a class with two members:

1
2
3
4
5
6

type SaveDivision() =
 member this Bind ((x:Results), (res: float->Result))
  match x with
  |Success(x) -> rest x
  | Infinite -> Infinite
 member this.Return x = x

At this point you can look at this as a class with two methods but what you really should see is that

• the map from a float number to the Result type is an amplifier or lift map as we described in the example above; it’s the endofunctor on the Net category of data types
• the Bind operation corresponds to the *-operation of a Kleisli triple
• the return corresponds to the eta-mapping of the Kleisli triple

and it turns this triple into a monad! You should also reflect on the fact that the Bind member correspond to the concat operation on the List category (and similarly the return corresponds to the return member).

The way you use the SaveDivision monad in a concrete computation is by instantiating it and do something inside as follows:

1
2
3
4
5
6
7

let safe = SaveDivision()
let SomeCalculation x1 x2 =
 safe {
    let! x = divide 1 x1
    let! y = divide 1 x2
    return divide x y
 }

and as soon as you try this on zero you get the Infinite as an answer. Without this construction you would get an exception, obviously.The ‘bang’ or exclamation mark after the let keywords is actually a way to tell F# that the Bind operation should be used instead of the normal assignment (or type inference).

So, this very simple example highlights the following:

• monads in F# allows you to do ‘stuff’ under the hood and redefine how things are being assigned or returned
• a monad in F# is really a Kleisli triple but because it’s mathematically equivalent to a monad there is no need to differentiate. I does confuse however when you try to compare the mathematical textbooks with the functional programming textbooks
• monads are often called workflows because it allows you do embed a whole logic or workflow inside the instantiated class (monad) and redefine how things like the assignment (let!) should function. You will also see that F# in fact has extended the notion of monad to include other monad members and in this fashion allowing you to redefine how for-next, try, yield and other statements are acting inside the monad.
• monads are sometimes called computation expression since it redefines how assignments and expressions should act on (category) objects

In Haskell you will not encounter the alternative names and the term monad is used uniformly.

The suspicious mind should by now worry about a lack of consistency and feel the need for more formal proofs that for example the constraints (or defining rules) of a Kleisli triple (aka monad) are satisfied. Indeed, is the SafeDivision monad with the specified operations satisfying the constraints? Well, yes, it does. But I will refrain to reproduce the proofs and am happy to forward you to someone you tediously wrote it all down; see Paul Abraham’s blog and this document in particular wherein you’ll find the F# proofs.

A simple dictionary

In Haskell the following type class is used to implement monads:

1
2
3

class Monad m where
   return :: a →m a
   (>>=) :: m a →(a →m b) →m b

You can immediately recognize the return and the bind methods but the symbol ‘>>=’ is usually used instead of ‘bind’. Purists will tell you that the discussion around monads in Haskell is not complete without the topic of ‘comonads’ and ‘cofunctors’ but it will take a while before these discussions appear in the F# community. See this article in case you want to know more about this.

In order to use monads in F# you only need to implement an (implicit) class which implements the Bind and Return member. The language extends the purely mathematical definition with eight extra methods:

Finally, the following table should be helpful in case you wish to move back and forth between F#, Haskell and the math literature.

Haskell	F#	Math
return	return
>>==	bind	* operator
instantiate the Monad type class	implement at least the Return and Bind members	NA
Monad	Monad, Monad Expression, Workflow, Computation Expression	Monad with an equivalent definition through Kleisli triple. The defining structure in F# and Haskell is really a Kleisli triple but no distinction is made there between the two types.
NA	NA
functor through a type class definition	usually not mentioned	functor
function	function (‘fun’)	morphism
Hask category of Haskell data types	Net category of data types	a wild zoo of categories; groups, topologies, graphs, differential geometries, quantum field theories and so on
composable functions	composable functions	binary operation and monoids

References

• F# monad library
• Haskell wiki
• Haskell category theory on Wikibooks
• F# programming on Wikibooks
• Try F#; awesome online F# learning site
• nLab on everything categories in mathematical physics

SharePoint Workspace was unable to interpret the SharePoint location. Please check and ensure the location contains no typing errors.

The solution to this consists in adding a public facing URL and making sure the server has remote differential compression installed/enabled.

Go to Server Manager > Features and take a look at whether the remote differential compression is enabled. If not, check and install.

Next go to SharePoint Central Administration > Alternate Access Mappings and click the ‘Edit public URL’ . Once there, make sure that besides the default internal URL there is also a public facing URL through which users can sync via the Workspace application.

Like most issues in SharePoint, fairly simple to fix but often a whole research before figuring out what is going on.

Recently a customer requested a particle simulation with the intention to investigate correlations and the effect of multiple ‘attractors’. The real, underlying issue in this demand is the fact that certain situations and systems are governed by laws (differential equations) which cannot be solved exactly and require a dynamic system to search for critical coefficients. These coefficients result in (semi) stable configurations or highlight boundary conditions inside which things make sense.

The problem with these kind of systems is the scalability which has two aspects; the visualization which doesn’t scale well with the retained graphics pipeline of WPF/SL/WP7 on the one hand  and the O(N*N) calculation of interactions between particles.

Regarding the visuals there is really a lot to consider:

• if you inherit from a baseclass like FrameworkElement or Control or even UserControl you carry around with each and every particle a whole lot of stuff; data binding, styling, event and other infrastructure. While all these things are really great in an LOB, one doesn’t usually need all this when dealing with (lots of) particles
• typically the underlying surface needs to be a Canvas but this element also brings along many gears; the retained rendering system, the measure/layout mechanics, styling and more. Maybe the bonus you miss when bypassing the Canvas is the ability to easily integrate zoom and pan in an application.
• how much user-interaction is required, are mouse-events necessary or is it a ‘let it run’ system? Here as well, you need to consider carefully the implications (especially related to the scalability of the app).

With respect to the simulation or calculation part things are more straightforward but you also need to think about

• the possibility to use F# for the more mathematically oriented parts. The disadvantage is however the need to constraint your model to make sure things talk to C#, or you have to go the full nine yards in F# (talking to XAML is still a disaster in this case).
• how to optimize the calculations and to make sure that inside loops you break early and branch as much as possible
• dropping unnecessary properties of the particles and make the particle system as lightweight as possible

Within the Silverlight/WP context my experience is that if you use the Canvas/UIElement combination you easily hit a wall around 500 particles due to the way the rendering engine is functioning and the fact that the UIElement (or FrameworkElement) are too ‘heavy’. The inter-particle calculation will still run OK with 500 particles but the end-result is that it’s not a good solution altogether.

One can keep the Canvas as underlying surface and render particles as inherited Shape items but this doesn’t push the boundary much.

A step further is to replace the Canvas with a bitmap and use direct pixel printing. This is easily done with the WriteableBitmapEx project you can find on CodePlex. This boosts your rendering to 5000 particles if you drop interactions. However, if you switch on inter-particle forces you hit a calculational wall around 1500 particles which is still way better than the retained rendering solution but still somewhat underwhelming.

So, as a next step it would be great to use parallel processing for the algorithmic part but unfortunately there is no task parallel processing library (TPL) for Silverlight (though somewhat made an unofficial port). Another possibility would be to use MailboxProcessor of F# but it would entail threading issues and an altogether more difficult system.

Various samples and libraries are available for XNA and Windows Phone which allow one to create particle effects but

• the often use sprites or similar techniques to minimize the particle count. The results are usually very good but the aim in the context of games is usually to create a certain effect rather than to give a physically correct simulation.
• they often do not exhibit inter-particle forces. If they do the particle count is low.
• they focus on effects and not physics

See also the Flint-sharp library which is a port of the Flash/Flex library.

The conclusion is that one can perform simulations and develop particle system in Silverlight if the need is not above, say, a thousand particles. Above this threshold things become more difficult and Silverlight as a medium becomes less suitable for this type of task. Obviously there are high-tech solutions on the basis of C++ and GPU instructions but this was outside my client’s scope (and my technical know-how as well).

Note: on different level you need to note in the attached simulation sample that

• the average speed varies a lot while the amount of interacting particles stays the same
• the ergodic phenomenon if easily shown; that the average velocity over the sample equals the average of one item in time
• it’s quite difficult to create stable configurations; which mimics well how the real world is also constantly influenced by quantum fluctuations

References

• TPL library ported to Silverlight 4
• Particle system for games (XNA & WP)
• Flint for C#
• WriteableBitmapEx on CodePlex

Most probably, if you receive this error you also will get it when opening the SQL Server Configuration tool with a message stating

It seems this happens when you run a 64bit version of SQL Server and have removed a 32bit one earlier. Both versions share the same WMI configuration and, hence, things get lost when uninstalling some stuff. In this Microsoft Support report you can find some background where the essential bit is:

mofcomp “%programfiles(x86)%\Microsoft\Microsoft SQL Server\100\Shared\sqlmgmproviderxpsp2up.mof”

(run this in an elevated  CMD) as well as reconfiguring the server to enable file streaming.

EXEC sp_configure filestream_access_level, 1
RECONFIGURE

(run this as a SQL query).

Lo and behold, if you go through this you can move on and use SQL Server WMI again. If you’re curious about mofcomp, it’s the managed object format compiler parser but very little info is available about this tool and it seems to me that it’s one of those things (like the registry) you better handle with silk gloves.