JavaScript and Services: JSON Server + JSON + JSONP // + MooTools // Discussions

We encourage users to post events happening in the community to the community events group on https://www.drupal.org.
yajnin's picture

UPDATE

This thread expanded into general discussions with JavaScript and Services. There are couple MooToolers here so many examples are MT oriented. However, the principals would carry to jQuery et al.

This thread will be maintained every couple months.

Original Post

////// JSON Server + MooTools Request.JSON

var jsonRequest = new Request.JSON({

  url: "http://yourdomain.com/services/json",

  onSuccess: function(response){

    var myDataArray = response['#data'];
    // Manipulate data as necessary

  }

}).send({ 'data': {
    'method': 'yourservice.new',
    'argument_1': 'your argument',
    'argument_2': 'your argument' 
  } 
});

// Class: http://mootools.net/docs/core/Request/Request.JSON

Comments

One challenge

yajnin's picture

This works beautifully, but last night I ran into a challenge in sending JSON strings. I'm working on it right now and will update once I know what's wrong. In the meantime, if anyone picks this up and has insight, I'd love to hear it.

Problem

I can store regular strings without problem. I can receive JSON strings from services/database, no problem.

I can ~send JSON strings, no problem. (The string appears fine in the post (via Firebug)).

No errors are reported.

HOWEVER, the JSON string does not write to the database via JSON server.

The exact same string DOES write to the database, if using the admin/build/services drupal interface. It appears as though JSON server is choking on the string somewhere during the process.

UPDATE + + + April 17,

yajnin's picture
      • UPDATE + + +

April 17, 2010

Using SERVICES 6.x-2.0 and JSON SERVER 6.x-2.0-alpha1, as well as JSONP

-- ( JSONP made possible only through a modification to the JSON Server module (see below))

This string issue is over turned. And my original solution (below) is no longer valid.

It threw me for a loop as my node.save was now not working.

The original solution (below) required the arguments to be sent as a string.

In my current scheme it now requires an object. (As originally expected.)

Why?

Looking at the module code I see there is a better parser and we are no longer relying on the parser built into Drupal.

I suspect this resolved my original problem. I haven't run various tests. Your mileage may vary.

There's growing energy in JavaScript methods and Services. This page includes information about JSONP and JSON Server.

This is something that I support, and there are various opportunities to voice your own support. If you are so inclined :)

See below.

Challenge Update

yajnin's picture

An attempt to be clearer:

I'm passing data from my JS app back and forth from the database. No problem.

However -- as well as sending regular string arguments in the POST object, I'd like to include a JSON string argument.

The regular arguments write to corresponding table fields in the DB as normal. ** The JSON argument (containing a lengthy string of fields) would write and be called to a single table field in the DB.

I'm running into a problem with the parsing of the JSON server data object. It seems to choke when it contains a sub object such as my JSON string.

I'm working on it. Any ideas in the meantime would be appreciated.

Update

Cataloging this process is helping me think through it. Here's what I've got so far...

After encoding the object into a string, it seems mandatory to escape the first curly bracket with another character. Anything other than ' [ ' or ' { ' which confuse the JSON parser.

So, as of now I'm writing strings like:

var myMash = "#{'foo','bar'}";

and it's working but not overly ideal.

Final (and easy) Client-side Solution

yajnin's picture

Here's the entire script to post to services via JSON server, using Javascript (MooTools). Data includes a few variables, as well as a JSON object string.

Included are two client-side options for solving the (current) problem of JSON string arguments not being parsed properly by JSON server.

//////// Set regular arguments

var title = 'Bob\'s Fish Diary - Day 82';

var body = 'Today I went fishing at the North Point and South Point. Check out what I caught below. – Bob';

var fish = {
  'trout':'at the north point',
  'perch':'at the south point',
  'trout':'at the south point'
};


//////// Set JSON argument

fishJSONstring = JSON.encode(fish); // This is a MooTools specific class, but similar functions are freely available

// We must escape the object and array brackets,
//   otherwise some part of JSON Server does not pass the variable properly to Services.
//   As of today, anyway :)

// -- Two options to escape that works with JSON server

// -- -- Slim (but dirty(it appears to work in this use case, but would need to review for others)?) single character escape

var fishJSONstringESC = '#'+fishJSONstring;

// -- Alternatively, you can use the (more proper?) encodeURI()

// var fishJSONstringESC = encodeURI(fishJSONstring);

// -- This option obfuscates and lengthens the data, which may or may not be desirable for you
// -- -- For example " { 'foo':'bar' } becomes %7B%22foo%22:%22bar%22%7D
// -- -- -- (Personally, one of my favourite things about JSON data is that it is naturally easy to read. )

// Final product

fish = fishJSONstringESC;

//////// Send Request (Using the MooTools Request.JSON class (link at top of this post))

var jsonRequest = new Request.JSON({

  url: "http://yourdomain.com/services/json",

  onSuccess: function(response){

    // console.log('success');

  }

}).send({ 'data': {
    'method': 'fishblog.new',
    'title': title,
    'body': body,
    'fish': fish 
  } 
});


//////// Unescaping Your JSON Object After
////////

// Assuming you've pulled the same data back using a 'get' service on the JSON Server:

// -- First Option (where you added "#" to the beginning of the fish JSON string)

var unescapedString = fish.replace( /#/, "" );  // This only removes the first '#', not subsequents

// -- Second Option (the encodeURI method)

var unescapedString = decodeURI(fish);

Update The problem this

yajnin's picture

Update, April 2010

    • The problem this solution was designed for appears to have been fixed.
      • Refer to my update comment for more information

http://groups.drupal.org/node/24372#comment-184103

mooTools encode vs serialize

mcaudy's picture

Hi ninjay,

I have no experience with mooTools, but it seems to me that you are mistaking "encode" with "serialize" functions. I think that what you want to do is serialize your JSON data to a string, not "encode" it.

If you google for "mooTools encode vs serialize" you will find quite a bit of discussion about this distinction. For example, the first two results from this google search are:

"JavaScript Form Serialization Comparison
In summary, mootools does not support semantic serialization and it incorrectly ... tested use JavaScript's encodeURIComponent function to encode form data. ...
malsup.com/jquery/form/comp/ - Cached - Similar - "

and

"PHP Serialize() & Unserialize Issues
If you encode the serialized array, you'll have to decode the array ... I'm a 26 year old Web Developer and jQuery & MooTools Consultant ...
davidwalsh.name/php-serialize-unserialize-issues - Cached - Similar - "

Hope this helps.

Mike

Nah

yajnin's picture

Hi Mike

Thanks for taking the time. I'm not mistaking anything.

MooTools' encode function takes a JavaScript OBJECT and converts it to a JSON STRING. The function I'm using is the correct one. Not knowing if you are familiar with JS, let me give you a quick example.

var myObject = {
  foo: 'bar'
};

var string = JSON.encode(myObject);

console.log(string);

// Returns -->    {"foo":"bar"}

The string that it returns is a JSON string. It's my expectation that this string should be able to be passed as an argument.

However, this fails, hence the need to escape the curly bracket.

Since writing this I've discovered that MooTools has a function to do this properly.

// string.escapeRegExp();

console.log(string.escapeRegExp());

// Returns -->   backslash{ "foo":"bar" backslash}
// (note I've written backslash as in the comment preview it seems that the slash is being dropped)

Escaping it is fine. I simply think it is an unexpected and unnecessary step.

node.save Sting/Object problem

abritez's picture

So my my next issue brought me back to the same thread. I think it is the same as what you experienced however i am getting an error.

Here is my request:

callback Request.JSONP.request_map.request_1
method node.save
sessid 123123123123123
node {"type":"blog","title":"New Title","body":"This is the blog body"}

Here is my result

HTTP/1.0 500 Internal Server Error

I got this working before, but i used AMFPHP and was able to send objects to drupal. I am guessing that this has to do with Drupal expecting an object, but since it is a GET it gets transformed as a string. Is there any way of getting around this with out hacking the code?

Here is my code:

 
       $('newBlogSubmit').addEvent('click', function()
        {
          var node = {
               type : "blog",
               title:"New Title",
               body :"This is the blog body"
            }
         
           var string = JSON.encode(node);
            string.escapeRegExp()
     
           var sessID = _sessID;
         
           DrupalService.getInstance().node_save(string, sessID, drupal_handleBlogSubmit);
        });

My Drupal Service JS Code:

//NODE

DrupalService.prototype.node_save = function(node, sessid, callback){
  var dataObj = {
        method : "node.save",
        sessid : sessid,
       node : node
    }
  DrupalService.getInstance().request(dataObj, callback);
}



//SEND REQUEST AND CALLBACK FUNCTION

DrupalService.prototype.request = function(dataObject, callback){
   new JsonP('http://myDrupalSite.com/services/json', {data: dataObject,onComplete: callback}).request();
}

If there is anyway around this please let me know. Once again, thanks so much

PS, decode

yajnin's picture

PS: Conversely, to decode a string is to convert a JSON STRING to a JS OBJECT. To anyone reading, these are the exact functions you would want to use to convert objects to strings and vice versa.

More issues then tissues :)

abritez's picture

Having an issue, and not sure what i am doing wrong. I keep on getting "Invalid method" as a result. Am i missing something?

Using: mootools-1.2.3-core-yc.js

window.addEvent('domready', function() {

    var jsonRequest = new Request.JSON({

    url: "http://mysite.com/?q=services/json",

    onSuccess: function(response){

    var myDataArray = response['#data'];
    var string = JSON.encode(myDataArray);
    console.log(string.escapeRegExp());

    }

    }).send({ 'data': {
        'method': 'system.connect'
      } 
    });

});

Charles Returns
error Boolean true
data String Invalid method

Tested Positive

yajnin's picture

Hey Alex,

I just responded via email with my solution. I see here that you've tested pretty much the same thing I've sent you.

Mine works fine however...

var jsonRequest = new Request.JSON({

          url: "http://domain.com/services/json",
          onSuccess: function(response){

            console.log(response);

          }
        }).send({

          data: { method: 'system.connect' }

        });

Hmm... I'm also using 1.2.3-core. Have you tested your services setup through the Drupal admin? Or any other service method?

Services version

yajnin's picture

Note that my module versions are Services 6.x-2.x-dev / JSON server 6.x-1.x-dev and I haven't updated in a while. That would be the uber suck to learn that it's broken with the newest dev release. Though of course there will be a workaround.

I'll test the newest releases and update the code if need be. Sometime in the next few days.

If you are still having trouble I can send you the older modules I have if you want to troubleshoot that way.

I will swap out my Services

abritez's picture

I will swap out my Services and JSON_Server mods with the one's you sent me, and will let you know how that goes. Should I expect this to work crossdomain (JSON/P), or was that not originally supported? I'm not totally sure on this, but i did test JSON server on the same domain that my drupal instance is installed on and it seemed to work fine. However, if I run the same script and make an HTTP request from outside of the domain then i start to see issues. I was hoping that it would be possible to pass (callback=CALLBACK) similar to the way that Del.icio.us handles it

http://del.icio.us/feeds/json/singpolyma.techblog/
http://del.icio.us/feeds/json/singpolyma.techblog/?callback=CALLBACK

As side note question regarding JSON/P: Since i am a bit of a noob in the Drupal world, how would Drupal deal with (?callback=CALLBACK) when cleanURLs are setup? Just curious and wanted to make sure that wasn't the source of my issues also.

Ahh. My solution is

yajnin's picture

Ahh. My solution is same-domain. I believe GET is officially unsupported and in the modules I sent you cross-domain definitely will not work. I suppose this is why you're having trouble. I just saw the JSON/P post below this comment. I'll add a bit of info there.

JSONP

nickvidal's picture

I modified the JSON server module to handle JSONP and GET/POST. When I was about to post the code, dmitrig01 asked me not to. Unfortunatelly my emails have gone unanswered and I wasn't able to convince him to update the changes to the JSON server module. Anyhow, the code lives here:

http://drupal.org/node/477012#comment-2050614

It's very basic and doesn't change anything fundamental. But if you need help, let me know.

Thanks, I will check this

abritez's picture

Thanks, I will check this out. Would i need to upgrade to Drupal 7 or is this code compatible with 6.14?

Compatible

nickvidal's picture

It's compatible with Drupal 6. Just be sure to change the json_server.info file appropriately.

Or, if you want, just make the following modifications to your current json_server.module file:

1) Replace all $_POST for $_REQUEST.

2) Changed the code so that if a callback is set, it will handle JSONP:

$callback = $_REQUEST['callback'];
if ($callback) {
header("Content-type: text/javascript");
return $callback ."(". $json .");";
} else {
header("Content-type: application/json");
return $json;
}

As you can see, these changes are easy and straight forward. I sure would love them to make it into the official JSON server module.

I saw the beginnings of a

yajnin's picture

I saw the beginnings of a process to support cross-domain but I haven't been following over the past couple months to know where the maintainer of JSON server stands. I must say... the official support for this module has seemed a bit weak, and communication has been terrible. Sorry to hear you've hit a road-block.

This weekend I intend to get caught up and perhaps I'll give your mods a shot as well.

#

To abritz and anyone else who's looking to use MooTools, my edit to the original class was to make it work for Post.

If Get is supported go ahead and use the original MooTools code swapping SEND for GET:

http://mootools.net/docs/core/Request/Request.JSON

MooTools also has a JSONP class:

http://mootools.net/docs/more/Request/Request.JSONP

#

I'll test the newest dev releases as well as nickvidal's changes this weekend and post back here with my working code.

Cross-Domain

nickvidal's picture

BTW, I've already tested cross-domain and it works as expected. The neat thing with GET support is that you can type the URL in the browser and see the result right there. Be sure to use content-type as text/plain when testing.

Hey Nick, I stumbled on this

yajnin's picture

Hey Nick,

I stumbled on this competing project Web Services. Essentially one of the biggest contribs to Services had a conflict in core ideology, and perhaps, a feeling of outpacing the Services maintainers and/or community.

http://drupal.org/project/webservices

There's a more blunt list of benefits list here:

http://drupal.org/node/305523#comment-1000959

A reasonable controversy, there's been a year's worth of discussion. I threw my 2 cents in and alluded to our recent discussion.

// ### UPDATE (May 4, 2010)

I just looked into this module again and discovered it's been unpublished by the Drupal security team. Good to know someone is looking out for this. Sad to know the project hit such a startling roadbump.

Thanks!

abritez's picture

Very appreciative for both of your your help just to sum up for anyone else's benefit.

Here is the MooTools JS i used:

window.addEvent('domready', function() {
$('myElement').addEvent('click', function(){
new JsonP('http://ExternalDomain.com/services/json', {
data: {
method: 'system.connect'
},
onComplete: function(data){
var results = eval(data);
alert(results["#data"].sessid)
}
}).request(); //alerts 8.3 - the rating of the ipod
});
});

Here is my edit as per nick's advice using json_server.module,v 1.6

function json_server_server() {
if (!isset($_REQUEST)) {
return drupal_to_js(array('#error' => TRUE, '#data' => "JSON server accepts GET requests only."));
}

//GET CALLBACK
$callback = $_REQUEST['callback'];

$methods = services_get_all();
$request = drupal_parse_json($_REQUEST['method']);
$args = array();


foreach ($methods as $method) {
    if ($method['#method'] == $request) {
        unset($_REQUEST['q']);
        unset($_REQUEST['method']);
        $args = array();
        foreach($method['#args'] as $arg) {
            if(isset($_REQUEST[$arg['#name']])) {
                $args[] = drupal_parse_json($_REQUEST[$arg['#name']]);
            }
            elseif($arg['#optional'] == 0) {
                return drupal_to_js(array("#error" => TRUE, "#data" => "Argument ". $arg['#name'] ." not recieved"));
            }
            else {
                $args[$arg['#name']] = NULL;
            }
        }

        $result = services_method_call($method['#method'], $args);

        if (is_array($result) && $result['#error'] === TRUE){
            return $jsondrupal_to_js(array('#error' => TRUE, '#data' => $result['#message']));
        }

        //ADD IN CALLBACK JSON/P SUPPORT

        $json = drupal_to_js(array('#error' => FALSE, '#data' => $result))

        if ($callback) {
            header("Content-type: text/javascript");
            return $callback ."(". $json .");";
        } else {
            header("Content-type: application/json");
            return $json;
        }      
    }
}
return drupal_to_js(array('#error' => TRUE, '#data' => "Invalid method $request"));

}

Glad to know your problem is

yajnin's picture

Glad to know your problem is solved Alex!

JSONP

yajnin's picture

IMPORTANT NOTE FOR ALL

As of November 2009

The code shown here is not an official contribution to the JSON Server module, as noted in the comments above. If anyone finds it useful, perhaps you can chime in on the discussion to have it recognized by the maintainer.

http://drupal.org/node/477012#comment-2050614

IMPORTANT NOTE - APRIL 2010

yajnin's picture

I'm using JSONP to exchange information easily across various domains / servers that are completely under my own control. This method is incredibly useful. There seem to be doubts as to what you would use JSONP for. There seems to be skepticism regarding 'security issues'. You are welcome to post questions here about this. I'll try and check in once every couple weeks.

JSON Server 6.x-2.0-alpha1

Notes for anyone attempting Nick's changes in 6.x-2.0-alpha1

    • I made the prescribed fixes (for example "$_REQUEST")

Everything works fine. Great.

    • Except GZIP

I didn't spend much time on this issue, so your mileage may vary...

It threw an error the first time I tried it with JSONP and in my test dev this functionality is unnecessary.

The GZIP functionality in 6.x-2.0-alpha1 -- I essentially commented most of that method out.

I don't necessarily advise doing this unless you are able to debug. As stated, I didn't spend much time and am not sure of the consequences :)

JavaScript - Node.Save / Edit // also // MooTools JSONP

yajnin's picture

I posted tips for:

JSONP + JSON Server Reference Information

yajnin's picture

Please refer to other discussions and JSONP examples:

Updates on this topic
++++++++++++++++

See Issues threads for further discussion regarding inclusion of JSONP in JSON Server

http://drupal.org/node/598358
http://drupal.org/node/624898

+

See tips for getting JSONP working with JSON Server 6.x-2.0-alpha1

http://drupal.org/node/624898#comment-2858192

// ##

See more discussion from people using it (as well as general JavaScript info, as well as MooTools info)

JSONP info begins about half-way down

http://groups.drupal.org/node/24372

// ##

See an example of using JSONP (with MooTools) for node.save, node.update

http://drupal.org/node/774116

Thanks for the updates on

backpacklife's picture

Thanks for the updates on this . However i'm still having difficulty understanding some of it.

I have been trying to find an example of a simple standalone html/javascript form that can submit a JSON/JSONP post to Drupal, as I think this would help illustrate the whole process to me rather than snippets. I've been googling a lot and still haven't found anything like this yet - so if you could point me in the right direction or be kind enough to show me this process I would be most grateful.

Cheers.

@backpacklife // Thanks for

yajnin's picture

@backpacklife // Thanks for reading

// //

Admittedly we assume a lot of knowledge in these posts. HTML and JavaScript are mandatory. Intermediate to advanced JavaScript.

// //

Once you've mastered manipulating data in JS, you just need to know where to put your strings.

It's all about the Node Object. Please refer here:

http://drupal.org/node/774116

// Note the keys "title" and "body"
// // These correspond to your form

/////////////////////

But what to do with a form that has different fields than "title" and "body"?

//

Well that's where a tutorial falls apart... there are so many options for so many use cases.

And so many consequences for each option.

//

For example:

The most common way to build a form in Drupal is with CCK.

You'd set your "type" key to the value of your custom content type. You would then add your CCK fields to your node object.

Sadly it's not as simple as adding:

"my_custom_field": "my_custom_field_value"

You'll need to know how those fields are structured (objects nested in arrays, and such) in the Node Object.

And you'll need to be mindful of CCK fields in Nodes or via Views. And then permission issues upon permission issues.

//

PHEW. I hope you're seeing my point. You'll have to work through at least 5 tutorials to really start to understand this.

And one would need to write at least 5 tutorials to teach it.

//

Over the next couple weeks I'll post about CCK fields. And access control.

And the (many) bizarre things discovered. Best I can do :)

Simple CCK Fields in the Node Object

yajnin's picture

// + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

UPDATE: This has been posted to http://drupal.org/node/776122

That version will be maintained more actively. Please consult there.

// + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

// Original Post

Required Reading:

http://drupal.org/node/774116

Understand these objects: "data" and "node_object"

Continue :)

///////// SIMPLE CCK FIELDS

In the code box below refer to:

custom_field_a = [{ value: 'custom field a' }]

// Which translates to:

// custom_field_a = [];
// custom_field_a[0] = {};
// custom_field_a[0].value = 'custom field a';

This is how simple CCK fields must be formatted.

// // CCK

var custom_field_b = 'custom field b'

data.node_object.custom_field_a = [{ value: 'custom field a' }];

data.node_object.custom_field_b = [{ value: custom_field_b }];

// // Title

data.node_object.title = 'New Post. Title: ' + custom_field_b;

// Etc

Refer back to // // http://drupal.org/node/774116

// + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

UPDATE: This has been posted to http://drupal.org/node/776122

That version will be maintained more actively. Please consult there.

// + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

thanks

backpacklife's picture

Brilliant. thanks for the quick reply!

As you say, I think i'm going to have to take a few steps back before I can tackle this issue head on. In my particular case i'm trying to use a non Drupal form to interface with Drupal - from another domain. Hence looking at services. This I see throws up a whole other can of worms with permissions etc and authentication. Fun! :D

What you want to do is

yajnin's picture

What you want to do is possible. As long as you setup a Drupal-side form that corresponds to your non Drupal form.

// Using CCK

Create your Content Type which allows you to create a 'form'. Define your fields.

You then can create new content, using the form you designed with CCK.

// Your Non-Drupal Form

Design your non-drupal form to correspond fields to your CCK form Drupal-Side.

As long as it matches what Drupal is expecting, voila.

This is where the "handshake" snippets come into play (in the above posts).

** BTW Cross domain posting

yajnin's picture

** BTW

Cross domain posting requires you to hack JSON Server, if you want to use JSONP. There are tips on doing that above.

But its not for the faint of heart. As any of this is not.

//

If you figure it all out -- its worth it. If you realize that the entire world is a "form" via JavaScript... its incredible what you can do with your networks of sites :)

Have fun!

Looking for some JSON help

jazzdrive3's picture

Looking for some JSON help when dealing with the views.get method. No matter what I have tried, I always get "Missing required arguments".

I've been able to grab nodes and session ID's just fine, but the views.get is making me pull my hair out!

      $.ajax({
           url: service_url,
          dataType: "jsonp",
           data: {"method" : "views.get", "view_name" : "rest_a" },
           success: function(data){
               if (data["#error"] == false) {
                  console.log(data);
                 $("#progress").toggle();
              } else {
                  console.log(data);
                 alert( "Error: " + data["#data"] );
                $("#progress").toggle();
              }
          }
      });

What argument am I missing exactly?! The only required one is "view_name" correct?

Thanks!