Creating a Navigable list of Property Values in Semantic MediaWiki

Update: I tweaked the code after posting this. It’s been updated to reflect my working solution!
One of the great things about Semantic MediaWiki (SMW) is the auto-generated fact box at the bottom of any wiki page that has any values for semantic properties.

Screen Shot 2013-05-01 at 10.25.12 AM

It’s also relatively easy to create a list of all defined values for a given property. In fact, this idea uses this template as a basis for what follows.

However, I found that it’s not very intuitive to navigate to a list of all values of a property and see what other articles have that same property value. In order to see a list of all articles that have a particular property value you’d have to navigate to an existing page with that property and then perform a semantic search from there. Or know that there’s a page called Special:SearchByProperty that you could use.

Screen Shot 2013-05-01 at 10.04.58 AM
Example of Special:SearchByProperty results

I wasn’t satisfied with that, so I made a fancy list that’s actually really easy to implement and can be used across an entire SMW for other property values too.

The result is a list of all values for a given property that provide hyperlinks to a list of all wiki articles with that property value. It uses the Special:SearchByProperty page for displaying results.

First, I created a template called Property Values Columns. You can use it like so:

{{Property Values Columns|<em>Name of Property Here</em>}}

Here’s the contents of the “Property Values Columns” template1:

<noinclude>Use this template to list all the unique values for a given property.

== Template code ==
</noinclude><includeonly>
*{{#arraymap:
{{#arraydefine: valued
|
{{#ask: [[{{{1|}}}::+]] | mainlabel =- | headers = hide |? {{{1|}}} | limit = 10000|searchlabel= }}
|,| print=list, sort=asc, unique}}
|,|@@|[[Special:SearchByProperty/{{Space|{{{1}}}/@@}}{{!}}@@]]|
<ul>
	<li>}}</li>
</ul>

</includeonly>

This is what it does.

1. Gets a comma separated list of property values for whatever property you’ve requested.
2. Parses the name of the property and each unique value to build a URL.
Example: http://wikiname.com/wiki/index.php/Special:SearchByProperty/Property Name/Property Value
3. The secret sauce is another template called ‘Space’ which uses the #replace function to replace all spaces ” ” with the “-20” syntax used in the Special:SearchByProperty query.
3a. The “Space” template contains the following:
{{#replace:{{{1}}}| |-20}}
4. Return the results as a series of list items.

The result is an list of all unique property values that when clicked will query the wiki for all pages containing that property value. You can then use some custom styling to format the results. In the example below I wrapped the template in a div creating two columns.2

<div class="twoColumns">{{Property Values Columns|Accounting Unit}}</div>

Screen Shot 2013-05-01 at 10.13.18 AM

Pretty cool huh?

Hierarchy in Properties Using Semantic Mediawiki #arraydefine and #arraymap Magic

At work, I’ve been working with our HR reps to develop something we’re calling Career Learning Maps.  The Learning Maps are a way of determining the various opportunities via career level (Positions such as Associate, Senior, Lead) and the skills required at each level. For each skill we’d have a list of suggested Learning Resources such as books, videos, seminars, etc. that assist in developing those skills.

For example, today you might be a Senior Database Engineer, but you’re looking to move up to Lead Database Engineer. What skills are required at this new level? Or what if you’re currently an Associate Technician, but want to move toward a Management position?

The Learning maps can help you understand the various job descriptions and associated attributes available to co-workers.

Since we already have Semantic Mediawiki as our knowledge repository for our department we decided to leverage its capabilities to deliver the Learning Maps.

One of the issues I was running into was figuring out a way to develop a hierarchical list of skills. Each skill is related to what we’re calling a Skill Dimension. Think of the dimension as parent groupings of skills. Business Acumen is a skill dimension that includes the skills of General Healthcare Industry KnowledgeProcess Improvement and Business Process Management among others.

I figured out a cool way to use #arraydefine and #arraymaptemplate to create a hierarchical list based upon properties. This list appears on each individual Job Description and shows the relationship between the career level, related skill dimensions (to that career level) and individual skills organized by skill dimension. Check this out:


Each job description is semantic form/template driven, as is each career skill dimension and career skill.

 

—-

Here’s the code – in case you might want to use it in the future.

In the Job Description Article Template (after building your form/template to include these properties!):

{{#arraydefine:levelsman|{{#arraymap:{{{Career_Level|}}}|,|@@|@@}}|,}}

(This defines an array using the specific job description’s career level.)

{{#arraymaptemplate:{{{Skill_Group|}}}|Career_Skills_Dimension_Template|,|
}}

(Ok, how many Skill Dimension apply to this job description? Give me a list and format it with a template.)

In a template called “Career_Dimension_Group_Template”

'''[[{{{1}}}]]'''<br>
{{#ask: [[Category:Career Skills]]
[[Skill Dimension::{{{1}}}]]
[[Career Level::{{#arrayprint:levelsman | }}]]
|link=none
|order=ASC
|format=ul
|default=No Skills Listed
|searchlabel=
}}

(Spit out a Skill Dimension, followed by an #ask that lists all relevant skills that are both part of that group and at the same level as this job description.).

The best part is that each property value is a page, so I can change “link=none” to “link=all” to have each skill in each group be a link to that individual skill’s article as well!

—-

To expand this even further, I’ve created a matrix listing all Skill Dimensions by Career Stream (Management, Professional, Technician) in a matrix with each Career Level. Co-workers can use this to easily see which skills are relevant across a given level or dimension.

While not very attractive, each cell in the matrix has a variation of the following #ask query.

{{#ask: [[Category:MTS Career Skills]]
[[MTS_Career_Level::Executive Director]]
[[MTS_Skill_Dimension::Strategic Focus (Management)]]
|link=all
|order=ASC
|format=ul
|default=
|searchlabel=
}}
The MTS_Career_Level and MTS_Skill_Dimension property values change for each cell.

—-

All together this creates a responsive and scalable solution to manage these relationships. Adding a new skill, resource or job description automatically updates all areas. In fact, most updating happens via forms and templates so our HR and Training folks can easily develop these resources with minimal Semantic Mediawiki development.

I hope this might be helpful for folks using Semantic Mediawiki to show relationships between properties in an interesting or unique way. Let me know if you have any questions or feedback.

Big Data, Mining, and (Musical) Recommendation Engines

As a side project in my free time I’m helping a small business setup an e-commerce store front. One of the things we’ve discussed is the idea of a recommendation engine to suggest other items to purchase. This lead down an Internet rabbit hole where I ended up reading about The Echo Nest.
The Echo Nest is a self-described “music intelligence platform that synthesizes billions of data points and transforms it into musical understanding.“. It is widely herald as one of the largest and most comprehensive uses of data mining (to find the language and culture around music across the web) and big data (to store and present those relationships) within the music recommendation industry.
Yes! There is an industry. A substantial one. Apple’s Genius feature in iTunes, Pandora, Last.fm, Spotify – all are trying to provide relevant music based upon your listening tastes. Why? So you’ll buy more music of course!
Brian Whitman, one of the co-founders of The Echo Nest, talks in great length about the how and why behind what makes their product so unique – and so incredibly accurate. I won’t steal the thunder of the article, but needless to say, dedication and refinement are key.
This is totally sausage-making, behind-the-scenes stuff, but I encourage you to at least look it over.
Ok, so now the really fun stuff. Here’s something called The Infinite Jukebox. It uses some of the data points within the Echo Nest to create a version of a given song that never ends. It uses references within a song that are similar to other points within the song, makes some minor adjustments when needed (like tempo) and then plays the song forever. The presentation is neat as well, you can view the branches within the song where things loop and even click around the song to find points where things can loop.

At work we’re looking at ways of using the topics of big data, mining, and recommendation engines to provide better healthcare. Reading about The Echo Nest gives me some ideas on how these technologies could impact the care we give! If you have your own ideas or suggestions, please leave a note below.

Semantic MediaWiki Templates and #arraymaps are Awesome

Templates are awesome.

As I’ve written about before, we use Semantic MediaWiki extensively at work. One way we use it is to handle research requests for or Solution Architecture team. We have a customer-facing form1 for all requests and the resulting page is accessible for anyone across the organization – sharing our findings beyond the original requester.

For any requests submitted, the form creates a new wiki article as a sub page of “Research”. This is done by adding an attribute of “query string=super_page=Research” to the form.

It helps us to keep things organized by denoting which pages are specific to research vs. general wiki pages.

The problem is how semantic queries display pages that have a ‘super page’ prefix. By default the query results will show the super page as part of the formatting.

Demo of a default query with no template

 

See the “Research/” prefix on every item? That’s rather redundant (and ugly), so I sought out a way to remove the ‘Research/’ prefix when displaying the results, but still provide the correct link to the sub page.

The magic is two-part. First, you need to make sure your #ask query has the attribute of “link” set to none (link=none) and “format” set to template (format=template)2. This strips out any default formatting of the results. Here’s the #ask query we’re using. Note you’ll obviously want to change the variables to fit your properties.

{{#ask: [[Category:Research]]
|?Research_Level_Requested
|?Research_Submitted_Date
|limit=15
|link=none
|format=template
|template=Research Results Template
|order=DESC
|default=No related research found. Submit a [[Research]] request?
|searchlabel=”’15 Most Recent Research Requests Loaded. View all Research?”’
}}

Then, for your template you’ll use the #arraymap function to format the output.

{{#arraymap:{{{1}}}|Research/|noSuperName|[[Research/noSuperName|noSuperName]]}} – {{{2}}} – {{{3}}}

What this does is for each result it removes the “super-page” prefix (in this case Research/) from the first property returned – the page name.

{{#arraymap:{{{1}}}|Research/|

It then replaces it with the variable noSuperName. 

|Research/|noSuperName|

Finally we actually construct a normal internal wiki hyperlink by adding “Research/” and the variable together in the proper syntax.

[[Research/NoSuperName|NoSuperName]]

The remaining variables {{{2}}} and {{{3}}} are called as normal and a break tag is added to keep each query result on its own line.

The result is something like the following screenshot.

Custom template sans-super page prefix

You will then have nicely formatted results that are easier to digest.

I hope this helps those looking to extend the semantic queries and produce clean, repeatable results. Let me know in the comments if you have any questions or ideas of your own.

Use Semantic Mediawiki & Semantic Forms to Create a Folksonomy for Tagging Related Pages

At work we use Semantic Mediawiki to augment an internal wiki running on Mediawiki. It’s used to house anything from process documentation to troubleshooting guides for our IT department. We recently figured out how to use Semantic Forms and the #ask function to create a customizable and reusable folksonomy. Read on to find out more.

—-

One of the functions of my team is to fulfill research requests for co-workers within our IT department. These requests can be as simple as something like finding a white paper from a vendor or research organization, or as in-depth as custom analysis and reports of a given topic.

In order to handle these requests, we’ve created a submission and request fulfillment process using Semantic Forms.

Co-workers can fill out the form and we’ll use the resulting wiki page to fulfill the request.

One of the fields in the form that we use when fulfilling the request is an open text box for tagging related topic areas. Those fulfilling the research request can use a comma separated list of items to generate a folksonomy that can be used elsewhere on the wiki.

In the form we have the following. The property “Research Related Tags” is a property with the type of “Page”.

{{{field|RelatedTags|property= Research_Related_Tags |values_from_property= Research_Related_Tags}}}

 

Then for our template, we have the following.

{{Research Entry Template|RelatedTags=}}

The following is to query the semantic data and display it.

{{#arraymap:{{{RelatedTags|}}}|,|x|[[Research_Related_Tags::x]]}}

The #arraymap function pulls back the list of tags and displays them in the template.


For example, I might get a request for researching more about Hover Cars. Hover Cars might be related to other wiki pages such as our Transportation page or a page titled Automobiles. If I enter a comma separated list of related pages into the tag box when fulfilling a research request (such as ‘transportation’ or ‘automobiles’, links to the research documentation will automatically be created to any page that matches that name.

Now the cool part is that we have a lot of existing content elsewhere in the wiki and we could never predict what new content is going to be created.

What we’ve done, is to modify the default template for every wiki page to pull back any research documentation related to that page. If you were on our Automobiles page at the bottom would be a link to any research requests tagged Automobiles. Automagically!

Silly nonsensical test items all tagged with “Big Data”

To do this, we use the #ask parser function to query the “Research Related Tags” property, but only show research requests that match the current page name.

{{#ask: [[Category:Research]] [[Research_Related_Tags::~{{FULLPAGENAME}}]]
|? Research_Level_Requested = Research Type
| ?PAGENAME= Entry Title
|format=ol
|limit=10
|link=subject
|default=No related research found. Submit a [[Research]] request?
|searchlabel=More Research Information
}}

The secret sauce is in this opening line.

{{#ask: [[Category:Research]] [[Research_Related_Tags::~{{FULLPAGENAME}}]]

This starts the inline query, limited to the Category of Research that has a value for “Research Related Tags similar (~ is a semantic wildcard) to the current FULLPAGENAME.

The rest of the ask command is pretty standard semantic media wiki syntax. The one additional item to point out is the default= condition. As I mentioned earlier, this query is on every wiki page and some (a lot of) wiki pages won’t have related tags.

If no research exists users are given the suggestion of submitting a research request. When new pages are created and they match existing research (or vice versa) this part of the page will automatically update with related research.

 

I hope this provides inspiration into a new way of extending the use of semantic data in your Mediawiki environment. Leave a comment if you have any questions.