Wandora D3 Graph Service....

[athanassios]

Happy new year,
I am about to make some concrete examples of how to use Neurorganon Upper Level Ontology (NULON) with Wandora. I would like to illustrate the design models with some nice graphs. I checked your D3 graph service and it looks very nice but unfortunately it is not synchronised with the Wandora graph topic panel. So first it would nice if it can be synchronised. Second it will be super if I can extract the D3 output and embed it on a HTML page for demonstration purposes. Then the idea of sharing or working collaboratively on topic maps, will be closer I think.

My wishes for peace and prosperity in 2012

PS: I have played a bit with the code on D3 but I have not managed to make it read my output e.g. in JSON format.....

[akivela]

Hi Athanassios

And very happy new year. I am afraid it probably takes some time while we adjust the look of Wandora's d3graph service. However, this rather long post helps you to create a static and Wandora independent graph visualization out of your topic maps.

To summarize required actions: First you need to create a new Wandora server module that will create a JSON data for the D3 graph library. This new module is based on existing d3graph module. Secondly you need to export your topic map as a JSON data and thirdly you need to create a simple HTML page using D3 which reads your JSON data and builds a graph visualization. Lets look at the details.

First you need to set up new server module in Wandora. This new server module should output JSON data that you can then use with D3. To set up a new server module browse
Wandora's directory build/resources/server. This directory contains a subdirectory named as d3graph. Duplicate it and rename the duplicate as d3graphjson. Open
a directory d3graphjson/templates and file 3dgraph.vhtml in a text editor. In text editor remove all lines except those that are related to D3graph JSON creation. To ease your work, next code fragment contains these remaining lines and you can just copy/paste the text into the 3dgraph.vhtml.

Code: Select all

#set( $wandoraClass = $topic.getTopicMap().getTopic("http://www.wandora.org/core/wandoraclass") )##
#set( $ctopic = $topic )##

#set( $topicMap = $topic.getTopicMap() )
#set( $topics = $topicMap.getTopics() )
#set( $ntopics = $listmaker.make() )
#set( $i = 0 )
#set( $topicSize = $topics.toArray().size() )
#set( $topicHashMap = $mapmaker.make() )
#set( $typeList = $listmaker.make() )
#if(! $request.getParameter("n") )
  #set( $n = 1000 )
#else
  #set( $n = $request.getParameter("n"))
#end

#set ($associations = $topicMap.getAssociations() )
#set( $assocTypeList = $listmaker.make() )
#foreach($association in $associations )
  #set($assocType = $association.getType() )
  #if(!$assocTypeList.contains($assocType))
    #set($temp = $assocTypeList.add($assocType))
  #end
#end
#set( $colorMap = $mapmaker.make() )
#set($colors = ["DarkGray","DarkGoldenrod","RoyalBlue","IndianRed","Gray","Violet","MediumAquamarine","YellowGreen",
"DarkSlateGray","SlateGray","CadetBlue","BlueViolet","Magenta","Brown","SeaGreen","SandyBrown","DarkMagenta","MediumSlateBlue","Orchid",
"Teal","LimeGreen","SlateBlue","SaddleBrown","Turquoise","DarkViolet","DarkKhaki","MediumVioletRed","Yellow","Black","DarkBlue","MidnightBlue",
"Tomato","GreenYellow","Gold","MediumPurple","Silver","Lime","DarkOrange","Green","MediumSpringGreen","Purple","Salmon","MediumOrchid",
"Moccasin","DarkSalmon","Coral","LightYellow","DarkOrchid","Beige","OrangeRed","MintCream","Orange","Cornsilk","SpringGreen","Maroon","LightCyan",
"RosyBrown","Azure","LightGreen","MistyRose","SkyBlue","PaleVioletRed","Lavender","DarkGreen","LightSkyBlue","DodgerBlue","DarkOliveGreen",
"DarkRed","Crimson","LightCoral","MediumSeaGreen","Seashell","Gray","Blue","Bisque","Peru","Pink","DarkTurquoise","SteelBlue","Olive","DarkCyan","DarkSlateBlue",
"Sienna","Navy","LightGoldenrodYellow","Honeydew","Indigo","Chartreuse","CornflowerBlue","DarkSeaGreen","OldLace","DeepSkyBlue","LightSalmon","PaleGreen",
"MediumTurquoise","PaleTurquoise","Goldenrod","FireBrick","Ivory","LawnGreen","Thistle","MediumBlue","LavenderBlush","BurlyWood","Fuchsia",
"Gainsboro","Aquamarine","BlanchedAlmond","AliceBlue","Linen","HotPink","Tan","OliveDrab","DimGray","DeepPink","Chocolate","ForestGreen","Khaki","Plum"])
#foreach($assocType in $assocTypeList)
  #set($j = $i % $colors.size())
  #set($temp = $colorMap.put($assocType,$colors.get($j)))
  #set($i = $i + 1)
#end

#set($i = 0)

{
"nodes" : [
#* *##foreach ( $topic in $topics )
#*   *##set( $topicName = "$topic.getOneSubjectIdentifier().toExternalForm()" )##
#*   *##set( $topicName = $topic.getBaseName() )##
#*   *##if($i != 0)
#*     *#,
#*   *##end
#*   *##set($temp = $topicHashMap.put( $topic.getID(), $i ) )
#*   *#{
#*     *#"name" : "$urlencoder.encode($topicName)",
#*     *#"id" : "node$i"
#*   *#}
#*   *##set($i = $i + 1)
#*   *##set($temp = $ntopics.add($topic))
#*   *##if($i == $n)
#*     *##break 
#*   *##end
#* *##end
]
,"links":[
#* *##set( $i = 0 )
#* *##set( $topics = $topicMap.getTopics() )
#* *##set( $doneAssocs = $listmaker.make() )
#* *##foreach( $topic in $ntopics )
#*   *##set( $assocs = $topic.getAssociations() )
#*   *##foreach( $assoc in $assocs )
#*     *##set($roles = $assoc.getRoles())
#*     *##if(!$doneAssocs.contains($assoc) && $roles.toArray().size() >= 2)
#*       *##if($topicHashMap.get($assoc.getPlayer($roles.toArray().get(0)).getID()) && $topicHashMap.get($assoc.getPlayer($roles.toArray().get(1)).getID()))
#*         *##if( $i != 0)
#*           *#,
#*         *##end
#*         *#{
#*           *#"source" : $topicHashMap.get($assoc.getPlayer($roles.toArray().get(0)).getID()),
#*           *#"target" : $topicHashMap.get($assoc.getPlayer($roles.toArray().get(1)).getID()),
#*           *#"class" : "assoc",
#*           *#"color" : "$colorMap.get($assoc.getType())",
#*           *#"id" : "link$i"
#*         *#}
#*         *##set($i = $i + 1)
#*         *##set( $temp = $doneAssocs.add( $assoc ) )
#*       *##end
#*     *##end
#*   *##end
#* *##end
#* *##if($topicMap.getTopics().size > 0 && $doneAssocs.size() > 0)
#*   *#,
#* *##end
#* *##set( $ntopics = $listmaker.make() )
#* *##set( $j = 0)
#* *##foreach ( $topic in $topics )
#*   *##set($temp = $ntopics.add($topic))
#*   *##set( $j = $j + 1)
#*   *##if($j == $n)
#*     *##break
#*   *##end
#* *##end
#* *##foreach( $topic in $ntopics )
#*   *##set( $types = $topic.getTypes() )
#*   *##foreach( $type in $types )
#*     *##if($topicHashMap.get($type.getID()) && $topicHashMap.get($topic.getID()))
#*       *##if($i > 0)
#*         *#,
#*       *##end
#*       *#{
#*         *#"source" : $topicHashMap.get($type.getID()),
#*         *#"target" : $topicHashMap.get($topic.getID()),
#*         *#"class" : "type",
#*         *#"id" : "link$i"
#*       *#}
#*       *##set($i = $i + 1)
#*     *##end
#*   *##end
#* *##end
]
}

And save the modified file to 3dgraph.vhtml. Next (re)start Wandora application and open NULON topic map. Then start Wandora's embedded server and browse new service d3graphjson. What you should now see in web browser is a JSON data you can use in D3 graph library. Save this JSON data into a local file named as data.json. This data file is next injected into a local D3 visualization.

Look at D3's web site at http://d3js.org/ and especially example Force-Directed Graph at http://bl.ocks.org/4062045 . Force-Directed Graph page contains an example code for index.html. Copy this example code into a text editor and change line

Code: Select all

d3.json("miserables.json", function(error, graph) {

to

Code: Select all

d3.json("data.json", function(error, graph) {

Now your text editor should contain a valid web page with a force-directed graph initialized with a JSON data file named as data.json. Modified web page code is

Code: Select all

<!DOCTYPE html>
<meta charset="utf-8">
<style>

.node {
  stroke: #fff;
  stroke-width: 1.5px;
}

.link {
  stroke: #999;
  stroke-opacity: .6;
}

</style>
<body>
<script src="http://d3js.org/d3.v3.min.js"></script>
<script>

var width = 960,
    height = 500;

var color = d3.scale.category20();

var force = d3.layout.force()
    .charge(-120)
    .linkDistance(30)
    .size([width, height]);

var svg = d3.select("body").append("svg")
    .attr("width", width)
    .attr("height", height);

d3.json("data.json", function(error, graph) {
  force
      .nodes(graph.nodes)
      .links(graph.links)
      .start();

  var link = svg.selectAll("line.link")
      .data(graph.links)
    .enter().append("line")
      .attr("class", "link")
      .style("stroke-width", function(d) { return Math.sqrt(d.value); });

  var node = svg.selectAll("circle.node")
      .data(graph.nodes)
    .enter().append("circle")
      .attr("class", "node")
      .attr("r", 5)
      .style("fill", function(d) { return color(d.group); })
      .call(force.drag);

  node.append("title")
      .text(function(d) { return d.name; });

  force.on("tick", function() {
    link.attr("x1", function(d) { return d.source.x; })
        .attr("y1", function(d) { return d.source.y; })
        .attr("x2", function(d) { return d.target.x; })
        .attr("y2", function(d) { return d.target.y; });

    node.attr("cx", function(d) { return d.x; })
        .attr("cy", function(d) { return d.y; });
  });
});

</script>

Save the text in editor to a file index.html. Save it to the same directory where you placed the data.json file.

And that is about it. Now you have a local HTML file that can be opened in a WWW browser. HTML file loads your JSON data into the D3 and visualizes the graph. As there is no more Wandora dependency, you can place visualization anywhere you like. And the best thing is that you can always create new JSON data files out of your topic maps.

Well, the visualization is not at all perfect. It doesn't contain node nor edge labels, for example. To make it look better, look at the documentation of D3 and Force-Directed Graph example. You might also want to examine render.js in Wandora's build/resources/server/d3graph/static/js .

Happy hacking,
Aki / Wandora Team

[athanassios]

Hi Aki,
I tested the template of the service you wrote, it works fine but as you said the result is very poor and needs a lot of experimentation with coding to set it right. Ideally it should produce at least the same visualization you get on Wandora graph panel.

Anyway thank you so much for starting it....

Athanassios

[Kergag32]

Is Wandora D3 Graph ok now? I have tried this template. But now I need it to be totally good. Sorry for asking to much.

[Anisorf]

Dear Akivela,

I'm modifying the d3graph service, I would like to make it interactive. So the nodes names of the graph will have hyperlinks. In the d3graph.vhtml file on line 131 (in my case) I added the url:

Code: Select all

"nodes" : [
#* *##foreach ( $topic in $topics )
#*   *##set( $topicName = "$topic.getOneSubjectIdentifier().toExternalForm()" )##
#*   *##set( $topicName = $topic.getBaseName() )##
#*   *##if($i != 0)
#*     *#,
#*   *##end
#*   *##set($temp = $topicHashMap.put( $topic.getID(), $i ) )
#*   *#{
#*     *#"name" : "$urlencoder.encode($topicName)",
#*     *#"id" : "node$i",
#*     *#"url" : "http://127.0.0.1:8898/topic/?$topic.getFirstSubjectIdentifier()"
#*   *#}
#*   *##set($i = $i + 1)
#*   *##set($temp = $ntopics.add($topic))
#*   *##if($i == $n)
#*     *##break 
#*   *##end
#* *##end
]


and add the xlink namespace

Code: Select all

<html xmlns:xlink="http://www.w3.org/1999/xlink">

Is this in your opinion good way to go ?

I should probably also change the render.js in order to make it work in something like this:

Code: Select all

 node.append("svg:a").attr("xlink:href", function(d){return d.url;})  // <-- reading the new "url" property
         .append("svg:cicle")
         .attr("class", "circle")
         .attr("r", 15);


Thanks in advance for all the support that you are giving me.

Best Regards,
Frosina

[akivela]

Hello Frosina

The URL value in the JSON data is missing the si variable name. The URL should probably be

Code: Select all

http://127.0.0.1:8898/topic/?si=$topic.getFirstSubjectIdentifier()

and the complete new line in d3graph.vhtml

Code: Select all

#*     *#"url" : "http://127.0.0.1:8898/topic/?si=$topic.getFirstSubjectIdentifier()"

And yes, you need to edit the render.js too. I figured a little different solution. Line 125 adds onmouse event handler to the graph node. One solution is to add click event handler to the node just after the mouseout handler. Remove the semicolon character ending the mouseout handler line and add click handler function:

Code: Select all

.on("mouseout", nodeMouseout)
.on("click", function(d) {
   //location.href = d.url;
   window.open(d.url);
});

You can use either the location.href or the window.open to open the url.

Kind Regards,
Aki

[Anisorf]

Dear Akivela,

Thanks a lot for the fast replay and for you solution. It works as I wanted, and in order to View the Topic in the web browser I've used this url for JSON in the d3graph.vhtml:

Code: Select all

#*     *#"url" : "http://127.0.0.1:8898/topic/?topic=$topic.getFirstSubjectIdentifier()"

Kind Regards,
Frosina

[akivela]

Oh yes, the url parameter name is really topic as you pointed out. Not si as I wrote in my reply.

Yet, I had problems with the solution in Firefox. Finally I got the link working with:

Code: Select all

#*     *#"url" : "/topic/?topic=$urlencoder.encode($topic.getFirstSubjectIdentifier().toExternalForm())"

Kind Regards,
Aki