MLJS 1.6 (Aug 2014) Released!

MLJS 1.6 (Aug 2014) has been released! Lots of features that are now even easier to use!

UPDATE: Try the Workplace App by following these instructions: https://github.com/adamfowleruk/mljs/wiki/Workplace-Application

Major new features include:-

• Complete UI redesign using Bootstrap CSS – allowing easy re-theming
• Completed Workplace widget – allowing drag and drop configuration of a page
• Created new Workplace Application – create an ENTIRE web app via drag and drop! ZERO JAVASCRIPT CODING REQUIRED!!!
• Created ‘extension points’ where developers can plug in their own widgets, page layouts, and search result renderers to extend MLJS without rewriting it’s code
• Updated Entity Facts widget to act as an InfoBox when combined with content search results (Shows semantic facts linked to subjects mentioned in first search result document)
• ‘Create’ widget now allows form creation and editing of XML documents via templates and typed XPath
• Major Node.js bug fixes and testing
• MLJS WebServer Node.js app to host REST API apps outside of MarkLogic server – reducing server load and increasing web app performance
• mlwatcher application – auto redeploys code or content to a MarkLogic database when files change on the file system

What’s the big deal

A long term aim of MLJS has been to lower the barrier of entry when creating a MarkLogic based application. MarkLogic’s Application Builder was great for using a web interface to create and configure an application, but going beyond a simple search page was difficult, requiring knowledge of several technologies and MarkLogic internal workings.

MLJS was easier to get started than customising Application Builder with just JavaScript code, but not as easy as a wizard driven interface. With the MLJS Workplace widget, and especially the Workplace Application, MLJS is finally as easy to use as application builder, but with many more features and ease of extensibility through just JavaScript.

To get jazzed about this, checkout the YouTube playlist – especially the Workplace Intro video.

What can I do now I couldn’t before?

You can now create and add your MarkLogic search options for your application and configure each page of your application within two minutes! I recently did a demo of creating a 5 page app within 5 minutes and 7 seconds.

Example uses include adding search facets, pager, sorter, search bar, and results widgets to a single page – perhaps with a chart or map too – to create a page like the traditional Application Builder page.

Alternatively, link a search bar, open layers map and search results widget for a simpler interface showing geospatial heat maps and search result markers. All this is configurable simply by linking these widgets to the same search context and options. Then you just need to tell the map the name of the constraint with marker information (a data series), and to render a heat map. The rest just works.

You can also create a semantic visual query page, with search bar, results, and entity facts widget showing all facts for a particular selected subject in the results. Perhaps you can even link this to another page of your application – when a user clicks on the ‘explore’ (all seeing eye) icon, another page is loaded showing a semantic graph navigation widget. This takes the page query string parameter for the Subject IRI in order to render it.

MLJS now comes with a page context object. This allows query string parameters to be extracted and set on key context objects if they exist on a page. This behaviour is on by default and can be overridden by configuring the page context object. This enables one page to link to others, passing information. It will also be the basis for future personalisation of the MLJS Workplace by users rather than page administrators.

I’ve also updated the create widget. Previously this only allowed file upload and read/update permission specification. Now it also includes a template mechanism for creation of new XML documents, or editing of existing ones. A flat, simpler alternative to full on XForms. Currently only XML support is added. JSON to be added in the future.

Last but not least I’ve got the automated testing working again. This threw up ten bugs with Node.js support that have now been fixed. There’s also an MLJS Web Server extension. This uses Node.js to host a MarkLogic web app outside of the MarkLogic server, making serving and caching web pages quicker without loading MarkLogic Server. This is used to deploy the Workplace and mljsrest test apps without using Roxy.

Why did it take so long?

At 4 months long this has been the longest release cycle in MLJS history. The reason for this is because I had to remove the code from the page creator in to a well tested and reusable set of widgets to allow visual configuration of an entire application. 324 new features, modifications, documentation edits, or bug fixes were done as part of this release.

There are now 23 end user widgets in the Widget API, 4 contexts in the Core API, and many REST API methods supported. The Core API now has 10497 lines of code, with the widget API at 18212 lines – including jsdoc documentation.

This release involved creating a whole host of ‘workplace admin’ widgets – basically the workplace UI elements. Examples include a generic JSON object configuration widget (config wrapper) with support for arrays and nested configurable JSON objects – a generic, very useful configuration widget that underlies everything.

I also had to create a drag and drop mechanism for re-ordering and adding new widgets to a page (drop zone). A layout framework with several example layouts. (column, thin thick and thick thin). These all use Bootstrap css layout classes, so should also work well on mobile devices.

I also needed a Workplace Context object to manage a page’s configuration, and a page context to pass in URL parameters as configuration overrides. Finally I needed a workplace navbar widget to show clickable links to any pages that existed in an application, and a workplace page list widget to allow adding and removing of workplace pages in a blank Workplace Application.

Then all this needed to be tested. The users of Workplace are likely JavaScript novices so won’t be capable of debugging things as well as the traditional MLJS user. Testing had to be rigourous – much more than ever before. Each configuration parameter needed to be tested, search options were all tested, and some missing configuration problems had to be handled gracefully. An example is the data explorer widget – previously this needed telling about every possible RDF type that was shown – now it performs basic rendering of unknown RDF types too.

The node.js based Core testing scripts were all fixed and used to test the Core API.

Now this is all done I’m moving back to two month release cycles, with interim one month bug fix cycles. The focus is going to be on little updates, often. Most functionality for most demos now exists in MLJS so it’s just a case of filling gaps and handling more advanced configuration scenarios.

How do I get started?

Best to watch the new MLJS Workplace play list on my YouTube channel. This provides a quick intro to creating an application as a showcase, then has individual tutorials for setting up workplace, and how to configure pages as examples.

Also don’t forget the WIKI page with quick links to documentation.

What awesomeness is coming in future!?!

The theme for Aug/Sep/Oct 2014 is Visualisations. This covers everything about showing data. I’m going to refactor the ‘data series’ concept to allow many sources to feed information to a set of data series objects.

Perhaps some data is from facet summaries, others from co-occurence, and others still from semantic fact values. MarkLogic 8 will be out in Fall 2014 too, so I’ll also include support for SPARQL aggregations in data series.

These data series will power not only the high charts charting widget, but also the open layers mapping widget, kratu table view widget, and an upcoming node graph widget.

The node graph widget will be different from the data explorer in that it’s visual look and feel will alter depending on data. Perhaps a particular colour for a type of subject, maybe it’s size will reflect the number of tweets a person sends or amount of money spent in a transaction. Distance between nodes may be the inverse of a co-occurence between both values.

Data may also be viewed on the widget from document values or other semantic facts against the subject rendered.

Another area I’m going to look at is the search results widget. I’ve recently added support for various sub-areas with a search results rendering – but I’ve not coded the different summaries yet.

These include embedding a list of extracted values, showing related semantic facts, exploring related subjects, showing a thumbnail summary, and action buttons for viewing/editing/deleting/adding results to a list – and plugging in your own actions.

I’m also going to start creating more admin orientated widgets. I’ve started work on a search options configuration widget already – supporting EVERY SINGLE OPTION in MarkLogic’s search API! (That one is for you Ganesh!)

I’ll also work on importing/exporting ontologies in the semantics triple config object. This’ll include allowing a user to set human readable text, and specifying ‘common name’ for a subject and it’s type for MLJS to render appropriately.

In future the MLJS WebServer will also link to an Alert widget and alert context, allowing MarkLogic alerts to be fired straight through to a web session in a browser in sub second response times. This code has been sitting around for nearly two years, but not yet integrated in to MLJS. Should be a quick win for rendering this data.

Finally, I’ll also include sample Workplace page configurations and examples for creating common pages. Perhaps you want a search page that looks exactly like Google, or a google images search result page, or even a WordPress like document summary list. These will be very pretty ‘art of the possible’ sample pages to give advice and themes that you can use in your own projects.

What if I have an awesome idea?

Then please tell me!!! Or better still, Fork the MLJS repo and create your own widgets. BE SURE to fork the DEV repo though! Same is true of all MarkLogic pre-sales and consulting staff – forking the DEV repo allows me to get bug fixes to you quick between release cycles. I also point blank refuse to accept pull requests to the master branch – I need to preserve the stability of the master branch. Once you’d added your new widgets or bug fix, send me a pull request for the DEV branch.