Comunica Tutorial - Basics

Comunica Tutorial

• First half: Introduction to Comunica
  • Comunica Basics
  • Hands-on tutorial: Querying with Comunica


• Second half: Advanced Comunica
  • Configuring Comunica with Components.js
  • Hands-on tutorial: Creating a Comunica module

Linked Data on the Web is complex

• Heterogeneous interfaces
• Multiple query languages and algorithms
• Many clients only support 1 combination
• Limits the combination of existing interfaces and adaptation of new ones

Comunica is many things

Comunica is a meta-query engine created to handle this complexity

Motivation

• Easily support extensions and variants of existing systems
• Modular system
• Semantic configuration

  Reusable experiments

Triple Pattern Fragments as an alternative RDF interface

• 2014: Querying Datasets on the Web with High Availability (Verborgh et al.)
• Alternative interface for exposing linked data
• Created a SPARQL client that supports these interfaces

TPF has many extensions

• Send allowed bindings to server

  Bindings-Restricted Triple Pattern Fragments (Hartig et al.)

• Use Approximate Membership Functions to filter bindings

  Opportunistic Linked Data Querying through Approximate Membership Metadata (Vander Sande et al.)

• Support substring filtering on the server

  Substring Filtering for Low-Cost Linked Data Interfaces (Van Herwegen et al.)

Different branches/clients to support these features

Modular system

• Quickly switch between different setups
• Change specific parts in the setup and keep everything else the same
• Compare the features you want to compare
• Easily add new features without having to rebuild other requirements

Experiment data

• Problem: repeat experiments, reproduce results, reuse new systems
• Wanted an easy way to share our setup and support reproducibility
• Eat your own dogfood: do this semantically

Modularity using the actor model

Logic is separated into different actors

Each actor independently performs a specific task.

• JSON-LD parsing actor
• Turtle parsing actor
• SPARQL union actor A
• SPARQL union actor B

Actors can be combined for solving more complex tasks

Actors are grouped in task-specific buses.

Mediators are responsible for selecting an actor to solve a certain task.

Example: Parse as fast as possible

Modules are wired together through semantic configuration files

Components.js: a semantic dependency injection framework.

Configuration files declare and parameterize actors, mediators and buses.

Links all the building blocks together, without having to do this in the code!

This will be covered more in-depth in the next presentation.

Implementation details

• Open-source under MIT license
• Implemented in TypeScript/JavaScript
• 114 separate modules (actors, buses, mediators)
• 1619 unit tests with a coverage of 100%
• Compatible with JavaScript API specification of RDFJS (W3C community group)

Current features

• Nearly full SPARQL 1.1 support Handled with an internal SPARQL algebra
• GraphQL queries GraphQL is translated to SPARQL algebra
• Federated querying of heterogeneous sources: Any combination of sources
• • Linked Data documents
  • Triple Pattern Fragments
  • SPARQL endpoints
  • HDT files
  • RDFJS Sources

Query from the Command Line

After installing the @comunica/actor-init-sparql package

Outputs the resulting bindings

comunica-sparql <datasource-url(s)> <query>
      

comunica-sparql http://fragments.dbpedia.org/2015-10/en \
            "SELECT * WHERE { ?s ?p ?o } LIMIT 100"
      

[
  {"?s":"ex:test","?p":"dbpprop:date","?o":"\"Summer 1998\""},
  {"?s":"ex:test","?p":"dbpprop:first","?o":"\"Raymond\""},
  {"?s":"ex:test","?p":"dbpprop:format","?o":"\"DjVu\""},
  ...
      

Set up a SPARQL endpoint

After installing the @comunica/actor-init-sparql package

Provides a functioning SPARQL endpoint

comunica-sparql-http <context>
      

comunica-sparql-http \
    '{ "sources": [{ "type": "auto", "value" : \
        "http://fragments.dbpedia.org/2015/en" }]}'
      

Query through your browser

Running Comunica engine in the browser

http://query.linkeddatafragments.org/

Host custom engines yourself: https://github.com/comunica/jQuery-Widget.js/

Hands-on: querying data using Comunica

Goal: Cover the different ways Comunica can be used to query data

This tutorial will introduce both command-line tools and code-based querying

No advanced JavaScript knowledge required, we start from scratch

Hands-on: requirements

• git: cloning the starting code
• Node.js (version 8.0 or higher): running the code
• A JavaScript editor (we will use WebStorm): editing the code

Hands-on: five steps

1. Querying from the command line: Query existing sources.
2. Querying from different sources: Create a local source.
3. Querying from JavaScript applications: Add querying to your application.
4. Setting up a SPARQL endpoint: Use Comunica as a proxy.
5. Setting up a Web client: Provide your own interface.

Hands-on: getting started

Each step is explained in detail on the GitHub wiki:

https://github.com/comunica/Tutorial-Comunica-Querying-Data

Tutorial Website: http://bit.ly/eswcquery