What is Orchestrate?

Orchestrate makes databases simple by powering full-text search, events, graph, and K/V storage behind a REST API.


Receive Blog Updates

Share:

Adron Hall

Symphonize Some Create, Read, Update & Delete [CRUD] via Orchestrate.js (2/3)

In the last article I provided a short definition of key value and graph data structures. Then followed that with some code using chance.js to generate some basic data, primed for use with orchestrate.io using the orchestrate.js driver. In this entry I’ll start off from there and get more data generated and added to the Orchestrate.io service. Then we’ll take a look at that data and how to retrieve and manipulate it.

First step, put some data in storage so that we can work with it. I added another file to the project that I started previously. In sample_crud_io.js I’ve setup a flow of CRUD.

var Chance = require('chance');
var orchestrator = require('orchestrate')("01233006-eaa7-4e3a-94d5-myguidkeyhere");
var chance = new Chance();

First I declared chance.js again (already added via NPM previously) and declared the orchestrate.js object. Note the require statement includes an additional value being passed for the Orchestrate.io security key. To find the key navigate to the Orchestrate.io Dashboard at http://dashboard.orchestrate.io. Click on one of the applications created.

Clicking on an Application

Clicking on an Application

Clicking on the application brings up the application collections, which will have the key in the upper right hand area of the page. Click in the text area where the key is obfuscated and it will display clearly. I had to hit ctrl+a to select the full key. As shown in the code above, I just pasted in the key to the value to create the orchestration object.

Getting your Orchestrate Key

Getting your Orchestrate Key

The next section of code I’ve created a helper function and the variables I’ll be working with. The first require object and faux Javascript instantiation take up the first two declarations. Requiring the chance library and then using new kicks off the randomizer for the object to use. After that I declare orchestrator with a require, using a second parameter that has the collection key. The last line is the actual collection name, which in this case is ‘listz’.

The function I setup because I don’t like to type new lines and I also like to move my output away from the edge of the console just a bit. Pick your poison for this section and however you’d like to see the output for your own readability.

var Chance = require('chance'),
    chance = new Chance(),
    orchestrator = require('orchestrate')("01233006-eaa7-4e3a-94d5-yourKeyCode"),
    collection = 'listz';

function displayIt(theDataToDisplay) {
    console.log('    ' + theDataToDisplay + 'rn');
}

The next section is where we kick off some generated data and add it to a collection to test out how all of these pieces work together with orchestrate.js.

The first thing I do is get the guid, this is what I’ll use for testing all phases of the CRUD with orchestrate.js. Next I use the displayIt function to display the guid to the screen so that I’ll have an active view into what’s going on.

Next I declare the inputData which will act as the value of the key value data that I will be writing to the collection. This is a simple set of JSON pairs, using chance to get some interesting data for the sample.

var key = chance.guid();
displayIt('Writing key ' + key);

var inputData = {
    "First": chance.first(),
    "Last": chance.last(),
    "Number": chance.d20() * chance.d10()
};
displayIt('...with value of ' + JSON.stringify(inputData));

// Writing the record and providing a success message or failure.
orchestrator.put(collection, key, inputData)
    .then(function(){
        displayIt('success!!');
    })
    .fail(function(){
        displayIt('Brutal, failed a lot eh!')
    });

Now that I have all that, I ran it once to make sure all my code bits were working ok.

$ node bin/sample_crud_io.js 
    Writing key CF65E6E6-8FD5-5AA7-BB5F-4538F93F92FC

    ...with value of {"First":"Lucy","Last":"Roberts","Number":112}

    success!!

Simply calling node bin/sample_crud_io.js the execution displayed the key and the sample value data to display, along with the success of the put. Now let’s see if we can add a get that will retrieve the data and a remove to delete the data we just put in the key value store.

In the code below I’ve added a get inside the promise with a respective remove nested in that. This shows the data getting created and added to the key value store, read from the store and then removed from the store.

orchestrator.put(collection, key, inputData)
    .then(function(result){
        displayIt('success!!');

        orchestrator.get(collection, key)
            .then(function(result){
                displayIt('data retrieved!');

                displayIt(JSON.stringify(result.body));

                orchestrator.remove(collection, key)
                    .then(function (result) {
                        displayIt('data deleted, all gone!');
                    })
                    .fail(function (err) {
                        displayIt('Another error ' + err);
                    })
            })
            .fail(function(err){
                displayIt('An error ' + err);
            });

    })
    .fail(function(err){
        displayIt('Brutal, fail a lot eh!')
    });

Now, one might ask, “where’s the the U in the CRUD?” Well, this is where we really get into the where there are differences in the traditional RDBMS CRUD patterns and the world of Multiversion Concurrency Control (MVCC). Let’s dive into a modern tool that provides a good example of this, git. Git provides the notion of a working copy, known as the HEAD reference.  In Git, the HEAD reference is the most recent revision to a branch in source control. Each revision prior to HEAD is marked by a commit reference. Ultimately a Git revision history is a “linear reference based storage.” The key value store in Orchestrate.io is a similar linear reference based storage, so that each additional put or delete is actually just storing a reference to a new data object. What this does, is make rollbacks, indexing, and geographic distribution much easier. But now that we have some context into what is happening underneath, that still leaves us without an immediate solution to doing an update. Or does it?

orchestrator.get(collection, key)
    .then(function(result){
        displayIt('data retrieved!');

        // edit some piece of the inputData here.

        orchestrator.put(collection, key, inputData)
            .then(function(result){
                displayIt('success!!');
            })
            .fail(function(err){
                displayIt('Brutal, fail a lot eh!')
            });

    })
    .fail(function(err){
        displayIt('An error ' + err);
    });

The practice when using the available verbs to gain a similar update type functionality is simply to do a orchestrator.get and then when done doing the change to the data value, just do an orchestrator.put value such as has been done in the code sample above. During this however, if someone else has retrieved the data object through a get and then done a put to write it back, they’ll have changed the data that I would be trying to write back with my put. In the traditional world of updates with an RDBMS you can set write-locks or other mechanisms to choose if last write wins or last write errors when writing back to altered data. In the orchestrate.io key value object, there’s a tool to provide a way to determine that decision with reads and writes.

Check out the API Reference for Orchestrate.io on Puts (Create/Upate). The section on conditional puts provides an avenue to do what would be perceived as an update. The idea of matching on the If-match or If-none-match for the reference key to verify if it has changed (someone else wrote before I did) or not (I’d be good to write back then) is how to determine and commit an update.

If you’re used to the world of RDBMS updates and realize that what is occurring in an RDBMS is actually a read, delete and then insert, you may notice that even this put that provides a way to update data is absolutely different than the expensive method used in an RDBMS system. An atomic update as an RDBMS commits the update is extremely expensive compared to the way reference pointer canonical updates are, in time and in resource cost. Even though after some time the key value store may hold more data, some in history, the difference maintained is extremely small in consideration of the often dramatic performance differences.

Summary

In this entry I’ve got a working create, read and delete code gist of data to and from the key value store. To complete the CRUD scope of work, I have also detailed how reads and writes are resolved with the orchestrate.io key value store. The features around If-match and If-none-match provide the solution around determining if a write succeeds or not. The next step is to generate data and run some query examples against the data store. Until next entry, happy coding!

– @adron