Wednesday, January 17, 2018

Darwino makes OpenNTF mobile!

[Latest News] The iOS app works perfectly, but we are struggling with the great fruity company to get it accepted to the store, because they claim there is not enough mobile content on OpenNTF... We'll do our best with the OpenNTF team to get more content leveraging mobile capabilities.

Darwino 2.0 features a mobile micro application architecture well integrated with the IBM Domino server. It can render any existing Domino web application (classic & XPages) on a mobile device while taking care of the painful details:
  • Real mobile applications available from both Google Play and Apple Store
  • Authentication management, to avoid prompting the user when the server requires credentials
  • Opening external pages in a separate browser window to provide the best user experience
  • Mobile notifications, to alert users when something new arrived and let him/her act appropriately
  • Plus many implementation details, like error management and recovery
To quickly experiment with the capability, we are providing a generic mobile client that one can use to connect to any Darwino enabled Domino server. This client is available on both application stores and is fully configurable to point to any server. As a default configuration, we make it point to OpenNTF!

As most of you know, the main OpenNTF applications, like Collaboration Today, are XPages based with a responsive design. This make them very easy to integrate with the micro application architecture. Moreover, Darwino extends Domino Designer with a new design element to detect database changes and send mobile notifications. With that in place on the OpenNTF servers, we are able to send new notifications when a new entry is available in Collaboration Today. By the way, these notifications use the mobile first IBM Cloud service (a.k.a. Bluemix).

The implementation on the OpenNTF server was straight forward:
  • Get the Darwino OSGi runtime installed
  • Set some configuration properties
  • Create a new design element for mobile notifications. To leave the existing databases untouched, we did that in brand new, separate database
Voila, in less than an hour it can be all setup. The same can happen to your own applications!

What are the next steps then? Well, a few things:
  • We need to work with the OpenNTF team to mobile enable other parts of the site. Ideally, the whole site should be mobile enabled!
  • Why not white labeling the generic client with a pure OpenNTF branding (icon, title, ...)? Darwino offers this options.
  • Move forward and create Darwino single page applications, using web or mobile native technologies, to get the data offline and provide the best user experience without connectivity!
Stay tuned, OpenNTF becomes mobile!

Tuesday, January 9, 2018

Dynamic document based security

Domino provides document based security through the use of readers and authors fields. This is a great capability that is fairly unique in the NoSQL industry heavily used by Domino developer. One caveat though: you have to explicitly manage the list of users/group/roles in each document, generally based on the document state or content. How many times did you write an agent (background task) to update these fields nightly?

Since the beginning, Darwino supports readers and authors (as well as excluded readers and authors to remove specific users from the authorized list). It is implemented by a hook in the SQL generator that systematically adds a condition to the queries. Here is how it looks like in pseudo SQL:
SELECT D.JSON FROM DOC D WHERE (....) AND (D.READERS IN (:userid, :userroles, :usergroups...)

userid, userroles and usergroups are values related to the user making the request.
All of this is implemented (and by the way optimized) at the core Darwino level. So it applies regardless of API you use: Java, JavaScript, REST services, GraphQL...

But in 2.1, it goes beyond this simple capability: it allows an application to insert its own SQL filter:
SELECT D.JSON FROM DOC D WHERE (....) AND (<custom condition>)

For example, the custom condition can be based on fields within the document. Let's suppose that the current user object has an attribute 'country' set in LDAP. Let's also suppose that the documents in a JSON store also have a field name 'country'. From a security standpoint, we only want to allow the users to see the documents tagged with their country. With classic readers fields, you'll need an agent that periodically updates the document readers, or use groups and assign users to groups. In both cases, there is some maintenance to do.

With Darwino, it becomes as easy as implementing a runtime extension that returns a condition for the current user (again, simplified pseudo code):

String getDynamicSecurity() {
    User u = getCurrentUser();
    return "{country: '"+u.getCountryAttribute()+"'}";


There is no need to maintain a list of readers here. If a user is moved to a different country, then the query will use the latest user country (or any other needed value), immediately!

The dynamic security condition can be expressed using the built-in, MongoDB like, query language as above. This platform independent query is transpiled to SQL by the core Darwino code.
But it can also be expressed as raw SQL, allowing the filter to be as complex as desired. For example, it can lookup in another SQL table, view or stored procedure for a list of authorized ids for the current user:
This is limitless. Let's suppose that you have a users database where each user document has the name of its manager. Now suppose that user documents can only be accessed by the user and his managers. Well, just get the list of authorized IDs from a recursive SELECT statement
If the subquery becomes too complex and leads to performance issues, you can create materialized views that will be managed by the database server.

Come on MongoDB, you cannot compete against the power of relational databases :-)

Thursday, January 4, 2018

Offline your data with React-Native applications

In a previous post, I mentioned how we are leveraging ReactJS to create state-of-the-art web applications. We are now moving a step forward by also leveraging its sister technology, React-Native, to create portable, yet native, mobile applications. With all the benefits of Darwino, which includes a local JSON store that replicates for the best offline experience, and all the other services provided by Darwino!

To make this as easy as possible, we are providing a simple service based API, that can be called from any native language (Java, Objective-C, Swift).

1- Darwino service API

The API is basically made of one class, exposed bellow in pseudo code:

class DarwinoApis {
  Object fetch(url, params, headers, content)

The fetch method calls the service identified by the URL and associated parameters, and returns the result. Behind the hood, such the API call will be either translated to a remote HTTP service call (online mode), or to a local API call (offline mode). This is done transparently by a dispatcher, so the developer does not have to worry about the current mode. It just works! This API can be called from *any* native application, regardless of the library/framework it uses.

This shows the beauty of the Darwino architecture: the services are all written independently from the HTTP stack, and thus can be executed locally without the need of HTTP calls. It makes them truly available offline, while providing the best possible performance as it also avoids the data serialization/deserialization needed by the HTTP transport.

Here is a basic architecture picture:

2- ReactNative integration

As the API is exposed using the platform native languages, it is easy to bridge it with ReactNative. Moreover, the bridge enforces the use of promises, thus enabling the new aync & await capability for the best developer experience.

Here is, for example, how to retrieve the number of documents from a database, regardless if it is local or remote:

async databaseCount() {     let url = ["$darwino-jstore","databases",this.getDatabase(),"documentscount"]     let count = await darwinoApis.fetch(url);     this.println("This database contains {0} documents",count);
Easy, isn't it?

Triggering the database replication isn't more difficult, see:

let result = await apis.replicate(this.getDatabase(),options); this.println("{0}",result);

3- Darwino API distribution

The APIs is provided as a library to be added to any existing application. On Android, it is a maven artifact that you can reference from your gradle build file. On iOS, it is a Cocoa framework that you can add to your XCode project. Just add these dependencies and you're ready to go!



4- Under the hood

Wait, Darwino is Java based, so how can we provide a native Objective-C or Swift API? How can we get all the Darwino services, built-in and customs, run natively on any device? As the goal is to have one single Java source code to ensure the consistency between all the platforms, so how is is exposed as an Objective-C API?

The magic comes from J2OBJC. Right from the beginning, we have been very careful to write "portable" Java, meaning Java that can be compiled or transpiled to many different environments (JRE, Dalvik/Art, RoboVM, Multi-OS Engine...). As J2OBJC made a lot of progress these past years, it became easy to get our code transpiled to J2OBJC. The result is a tiny, efficient, native Objective-C library while sharing the same Java source code.
With the potential to target other platforms in the future, like .NET: what about having your Domino data replicated natively to .NET? :-)

Tuesday, January 2, 2018

Darwino 2.0 is out!

I'm very exited to announce that the official Darwino 2.0 has just been released!

It had been a long journey since the previous release but most of our customers are already using the pre 2.0 release, aka 2.0.0-SNAPSHOT in maven terms. This means that this release has already been tested in many contexts and is thus very stable.

The main topics for this major release are:

  • Creation of new web and mobile applications
    - Provide the best user experience even when the device is offline
    - Leverage of the latest technologies, including ReactJS
  • Modernization of existing applications, in particular Notes & Domino
    - Liberate the Domino data and take advantage of any data reporting/analytics tool
    - Get your existing Notes/Domino apps on mobile device with notifications
    - Create modern mobile applications connected to your IBM Domino data
  • Migration of IBM Notes/Domino applications
    - Incremental migration to minimize the risk
    - Use of modern technologies and development tools
    - Integrate with MS Sharepoint, Office 365 or other platforms, like Oracle Cloud
  • Many additions and enhancements to the existing APIs

I bet that the mobile client is going to be popular in our community because... Well, read my next post :-)

Bellow is a list of the most noticeable features in 2.0:

Darwino for mobile devices

Darwino mobile client
Darwino provides a generic client that renders server side web applications and handles IBM Cloud notifications. This client is available for both iOS and Android, in their respective application stores.
This client can also be fully customized as the full source code is available. For example, one can white label the application with a custom icon, look&feel, options...

Multi-OS-Engine instead of RoboVM
Darwino switched to MOE as RoboVM is no longer available since it has been acquired by Microsoft. Darwino 2.0 supports the latest version of MOE. Thanks Migeran for doing this great product!


Notification framework
Darwino provides a notification framework that executes actions based on events. The framework can detect data changes in a database, including domino databases, and run actions like a mobile push notifications, notifying IBM Workspace, sending an email or whatever a piece of code can do...

Microsoft Azure active directory support
Darwino features a driver to consume Microsoft Azure active directory services

Microsoft Sharepoint integration
Darwino integration with Sharepoint, including directory access and single sign-on.

More database support
The MS SQL server support is complete, as well as MySQL (the full JSQL requires MySQL 8+).

Darwino DB

JSQL is an SQL dialect dedicated to JSON databases. It allows the full power of SQL queries on top of JSON documents: joins, unions, groups, recursive queries, ... while honoring the document level security. It eventually translates to the native database SQL for maximum performance.

Registered queries
Database queries can be defined on the server side and consumed by name from the client. This is also true for JSQL queries.

Domino @formula
An extension to the query language uses the new keyword $atFormula and allows the use of a subset of the Notes/Domino @formula language in Darwino queries. That makes the migration of Notes/Domino views a lot easier!

REST services

JSON store services
Added some new services to retrieve document content without the meta-data.


Darwino comes with a innovative implementation of FaceBook's GraphQL, allowing both schema based and schema less queries. Moreover, GraphQL queries can be pre-defined on the server side to make them easier to consume

Darwino provides a easy to use JSON microservice infrastructure. The services can be written in Java, Groovy, JavaScript, Darwino Script or any language supported by the JVM.

Darwino Studio

Application Wizard
The application creation wizard can now use templates, also known as boilerplate, to get the developer started quickly. A new template using ReactJS and Bootstrap is provided as part of the studio. The architecture is open and any developer can contribute new templates.

Android studio
The wizard can also leverage Android studio for the mobile applications, instead of Eclipse. This allows the developers to benefit from the latest features provided by Google and Multi-OS-Engine.

UI Builder
Darwino studio features an easy form builder that targets mobile devices. This builder is fully extensible to support more targets in the future.

Darwino for IBM Notes/Domino

Domino Designer
Darwino provides a new Domino Designer plugin that adds a new design element dealing with the notifications. It uses a groovy based DSL (Domain Specific Language) .

Darwino client
The Darwino mobile client detects if the server is an IBM Domino server and adapts some behaviors accordingly, like error handling.

Darwino application
Darwino provides new runtime capabilities to supplement IBM Domino missing features, like JEE filters. Moreover, Darwino applications can be auto-started, for example to monitor events.

IBM Domino import
The Darwino studio can generate a basic UI based on existing IBM Domino NSFs. This feature is a preview, but this is something we'd like to enhance moving forward, to get the mobilization or migration of Notes/Domino import easier.
It also generates JSON Schemas based on the Form definition in an NSF.

Developer tools

Commands framework
Darwino features a command line framework that developer can add to any application to execute administration commands or debug an application.
It also includes a built-in application profiler.

The Playground now show cases the new features like JSQL or GraphQL, but also exposes the REST apis using OpenAPI/Swagger: