I've searched the web looking for something more than a "create a node" example for Drupal Services. There are some good examples for Flash/Flex and some bits and pieces in Python or PHP but nothing that helps me to figure out what I am doing wrong.
Since my client is planning to develop a cross platform application that interacts with their Drupal site, I thought I would create a sample app in Python - just to show the Flash, iPhone, Droid, Blackberry developers how the Drupal Services API works.
I may be doing something dumb. I've been programming for over 20 years but I have never really done anything serious with Python. I have a really hard time mapping what I know in PHP onto Python.
But, since the code will be of use to others and I really want to figure this out, I am posting it here and asking for help with one area of the sample app. I can't figure out how to attach the file upload to the node. I think it's because I am not passing the data correctly. Has anyone made this work when calling the XML-RPC server?
#! /usr/bin/python
import os.path, sys, time, mimetypes, xmlrpclib, pprint, base64
pp = pprint.PrettyPrinter()
config = {
'url': 'http://example.com/services/xmlrpc',
'username': 'developer',
'password': 'bZdKNTv7XB8C',
}
# Make initial connection to service, then login as developer
server = xmlrpclib.Server(config['url'], allow_none=True);
connection = server.system.connect();
session = server.user.login(connection['sessid'], config['username'], config['password']);
sessid = session['sessid'];
user = session['user'];
timestamp = str(int(time.time()))
## Load a movie file - get size, mimetype
filename = 'testfile.MOV'
filesize = os.stat(filename).st_size
filemime = mimetypes.guess_type(filename)
fd = open(filename, 'rb')
video_file = fd.read()
fd.close()
# Create file_obj with encoded file data
file_obj = {
'file': base64.b64encode(video_file),
'filename': filename,
'filepath': 'sites/default/files/' + filename,
'filesize': filesize,
'timestamp': timestamp,
'uid': user['uid'],
'filemime': filemime,
}
# Save the file to the server
try:
f = server.file.save(sessid, file_obj)
except xmlrpclib.Fault, err:
print "A fault occurred"
print "Fault code: %d" % err.faultCode
print "Fault string: %s" % err.faultString
else:
pp.pprint(f) # DEBUG
# Get the new file from the server (verify) DEBUG
try:
ff = server.file.get(sessid, f)
except xmlrpclib.Fault, err:
print "A fault occurred"
print "Fault code: %d" % err.faultCode
print "Fault string: %s" % err.faultString
else:
# We have the new file info - we need this in case the name of the file was changed when sotred on the server
# pp.pprint(ff) # DEBUG
del ff['file'] # We don't need this - free it
node_files = {
'fid': f,
'filemime': filemime,
'filename': ff['filename'],
'filepath': ff['filepath'],
'filesize': filesize,
'status': '1',
'timestamp': timestamp,
'uid': user['uid'],
}
# Create the node object and refernce the new fid just created
node = {
'type': 'video_local',
'status': 1,
'type': 'video_local',
'title': 'Remote Test ' + timestamp,
'body': 'This is a test created from a remote app',
'uid': user['uid'],
'name': user['name'],
'changed': timestamp,
'files': {},
'field_video_overlay' : [
{'value': 'Here is some CCK field text'},
],
}
# my latest attempt to figure out how to get node->files in the right format for the node.save method
# please don't laugh - I am just grasping at straws here - ive tried all the lists and dictionary formats.
node['files'][f] = node_files
pp.pprint(node) # DEBUG
try:
n = server.node.save(sessid, node)
nn = server.node.get(sessid,int(n),[])
except xmlrpclib.Fault, err:
print "A fault occurred"
print "Fault code: %d" % err.faultCode
print "Fault string: %s" % err.faultString
else:
pp.pprint(n) # DEBUG
pp.pprint(nn) # DEBUGThe code formatter makes it appear that some of the code is not indented correctly. See the attached file for working code
This code runs without errors - connects, logs in, uploads a file, gets the fid, creates a node - but in the end, the new node does not include the file attachment.
Does anyone know what I am doing wrong?
| Attachment | Size |
|---|---|
| drupal_services_test.py_.txt | 2.82 KB |
Comments
output from the pprint debug code
{'body': 'This is a test created from a remote app', 'changed': '1263608390', 'field_video_overlay': [{'value': 'Here is some CCK field text'}], 'files': {'534': {'fid': '534', 'filemime': ('video/quicktime', None), 'filename': 'testfile.MOV', 'filepath': 'sites/default/files/testfile.MOV', 'filesize': 893292L, 'status': '1', 'timestamp': '1263608390', 'uid': '9'}}, 'name': 'developer', 'status': 1, 'title': 'Remote Test 1263608390', 'type': 'video_local', 'uid': '9'}{'body': 'This is a test created from a remote app', 'changed': '1263608353', 'comment': '0', 'comment_count': '0', 'created': '1263608353', 'data': 'a:1:{s:13:"form_build_id";s:37:"form-b25e97816649fb6a7b49a7f115227c65";}', 'field_video_attach': [''], 'field_video_overlay': [{'value': 'Here is some CCK field text'}], 'files': [], 'format': '1', 'language': '', 'last_comment_name': '', 'last_comment_timestamp': '1263608353', 'log': '', 'moderate': '0', 'name': 'developer', 'nid': '99', 'picture': '', 'promote': '1', 'revision_timestamp': '1263608353', 'revision_uid': '9', 'status': '1', 'sticky': '0', 'taxonomy': [], 'teaser': 'This is a test created from a remote app', 'title': 'Remote Test 1263608390', 'tnid': '0', 'translate': '0', 'type': 'video_local', 'uid': '9', 'vid': '99'}I am running the script from my laptop and connecting to a Drupal site on a different machine.
You are mixing up attachments
You are mixing up attachments and file objects. Attachments have different properties, see upload.module, line 410 ff. Since you are creating the file beforehand, you only want to save a new attachment with the node. Without any prior Python knowledge, node_files should IMO look more like this:
node_files = {'new': 1, # Required to insert the referenced file->fid as new attachment!
'fid': f,
'list': 0, # Or 1, depending on whether you want the attachment listed on node view.
'description': 'whatever',
'weight': 0,
}
-Stefan
That's it!
That's it! I plugged in your code and ran my script. The uploaded file appears in the File attachments section as I was hoping.
{'body': 'This is a test created from a remote app', 'changed': '1263691037', 'comment': '0', 'comment_count': '0', 'created': '1263691037', 'data': 'a:1:{s:13:"form_build_id";s:37:"form-b25e97816649fb6a7b49a7f115227c65";}', 'field_video_attach': [''], 'field_video_overlay': [{'value': 'CCK field text goes here'}], 'files': {'535': {'description': 'Video Attachment', 'fid': '535', 'filemime': 'video/quicktime', 'filename': 'testfile_0.MOV', 'filepath': 'sites/default/files/testfile_0.MOV', 'filesize': '893292', 'list': '1', 'nid': '100', 'status': '1', 'timestamp': '1263691075', 'uid': '9', 'vid': '100', 'weight': '0'}}, 'format': '1', 'language': '', 'last_comment_name': '', 'last_comment_timestamp': '1263691037', 'log': '', 'moderate': '0', 'name': 'developer', 'nid': '100', 'picture': '', 'promote': '1', 'revision_timestamp': '1263691037', 'revision_uid': '9', 'status': '1', 'sticky': '0', 'taxonomy': [], 'teaser': 'This is a test created from a remote app', 'title': 'Remote Test 1263691075', 'tnid': '0', 'translate': '0', 'type': 'video_local', 'uid': '9', 'vid': '100'}Thank you smk-ka. You rock!
Media Mover takes over from here
In my current situation, I use Media Mover to detect the file as a file attachment, transcode it and attach the result to the field_video_attach CCK field. So this solution works great for me.
If, in the future, I want to attach the file directly to a CCK field, do I use the method that I was attempting in my original solution? And actually pass the file as data with the node? You imply something to that effect in your comment about my confusing attachments and file objects.
Cheers,
-ped-
No, this doesn't work with
No, this doesn't work with XML-RPC requests, since upload.module expects the file data in a separate variable (the $_FILES array in PHP, which isn't available in XML-RPC calls). Therefore the chosen way is the correct solution.
-Stefan
Can this be done via Services
Can this be done via Services instead of XML-RPC to bypass all of the media mover work?
thx
Cool stuff! I'm looking to do
Cool stuff! I'm looking to do this too. Can you please explain how you configured media mover to handle this?
Thanks!
Media Mover
The python code is just one example of how to create a node from remote and attach a file upload. Anything that you do in MediaMover will work the same whether you create the node from the web interface or from a remote app.
Your question is about Media Mover and so we should really be posting that in a Media Mover thread.
On the face of it, the process is quite simple. In reality, there are issues beyond just Media Mover (see below). Media Mover has built in support for harvesting attachments from nodes, processing them, and attaching them either to a new node or to the same node. In my case, I harvest the attachment, process it with Ffmpeg, and then re-attach it to the same node in a CCK filefield. Nothing special there. The only tech note I can offer is that there is a bug in the version of MM that I am using (i'll get around to fixing it soon) that causes MM to re-process all attachments every time it runs. It should only process new attachments, but I get around this by simply telling MM to "Delete original source" file but that is not ideal. See this bug report: http://drupal.org/node/645060
While this is a very natural use of Media Mover, it does presume that you have Ffmpeg set up correctly on your server and you know how to display the resulting FLV file in the node (from CCK filefield). Try http://drupal.org/project/swftools
In my current project, we are doing some unique things that are specific to our needs, but I originally got started with some very useful insights from this post: http://drupal.org/node/465230 and the Media Mover project page: http://drupal.org/project/media_mover
I spent hours configuring and compiling Ffmpeg for my platform but never got exactly what I needed (codecs, etc) http://ubuntuforums.org/showpost.php?p=8345112&postcount=636
so it was a huge relief when I found this static binary http://24b6.net/content/ffmpeg-binary provided by arthurf
Media Mover is still in development. If you stick to what works, you should be fine. When you get very creative with it, as I have had to, it is very fragile. Check the issue queue to see what I mean. http://drupal.org/project/issues/media_mover
Cheers,
-ped-
Thanks for the info! It's a
Thanks for the info! It's a bit scary that Media Mover is still in dev, I've already experienced some issues...but will see if I can get it to work.
Please help.
Hi,
I'm basically doing something very similar to what you are doing, and based on your website, you may be specifically interested in my project, it is a unique opportunity IMHOP.
At the very least, could you please contact me, so we can discuss what i'm doing and maybe answer some of my questions either here or through email.
I am working with a commercial vendor to create a drupal plugin to integrate a media encoding server that can extract text from videos using untrained speech to text processing. I.e. upload a video and outcomes a transcript that is probably about 50% accurate, but sufficient for searching. There are some other issues with this, but that we can talk about that offline.
I've developed with the services module a while back but I forget some things and need to do some new things.
THe media encoding server will send a message to Drupal that pushes the file to Drupal. Based on this, it may need to be in the $_files array of an HTTP POST request.
At that point the file should be sent to Drupal
Upon completion of a successful transfer the system needs to notify the encoding server that the file is transfered correctly and that the node is created.
Is it going to work something like this. The request data will be a json text in the request and then the $_files will contain the file. The $_files will be accepted and the data of the json text will be used to communicate information to the server about the video file?
We are creating a CCK customized field that can be added to nodes like file field; however, there is a lot of information related to the xml text transcript and xml video meta data file, which just is outside the scope of file field and needs to be handled differently IMHOP.
We are basing our module on file field though.
Please let me know if there is anything you think I should know about doing this, or anything you learned, since you solved this problem.
Services, Drupal 7
Hello.
Can anybody say about Services and Drupal7?
I have been working with 6 for a long time.
But I can not font any useful information for 6.
Thank you.
Dear Viktor, As I understand
Dear Viktor, As I understand you cannot find any information about services 7.x. You are very welcome to use them as they are more or less ready for usage. Please let me know whether you might have any specific questions about them.
Thank you for your reply. I
Thank you for your reply.
I have seen your video from DrupalCafe Ukraine once more.
My problem was That I had not understood new features of the Services in Drupal 7.
I did not know nothing about "end points". It's cool feature. It adds a new server layer. So I can able to communicate with Services using different client protocols.
I've added a new end point, determined the type of the server, enabled resources and actions.
I do not creating new service modules, I use the existed...
I see, there is not the stable version. I had got the problem: "undefined function drupal_set_header", I see that this functios is only used in Drupal 6.
So I use the last dev version.
Yes. The best is to use
Yes. The best is to use latest dev version for 7.x. I don't quite understand how I can help. Please contact me on the IRC (#drupal-services) or skype if you need any further assistance. And of course you are welcome to come to next Drupal Cafe to talk in person :)
Hi guys, hope this helps!
Hi guys, hope this helps! http://dgtlmoon.com/drupal_7_xmlrpc_python_example
http://dgtlmoon.com
I want to admit that the
I want to admit that the "file.save" does not exist any more.
Insted of it the file.create is? But I can not get function signatures.
I see empty help.
When I try add a file I get the error.
"Ошибка сервера. Подпись запрошенного метода file.create не определена."
"Server error. The signature of file.create method is not underfined"
Is it right ?
return self.server.file.create(self.session["sessid"], file_obj)
I try so:
def send_file(self, path):
file = self.load_file(path)
timestamp = str(int(time.time()))
file_obj = {
'file': base64.b64encode(file["data"]),
'filename': file["filename"],
'filepath': 'sites/default/files/tmp/' + file["filename"],
'filesize': file["filesize"],
'timestamp': timestamp,
'uid': self.session['user']['uid'],
'filemime': file['filemime'],
}
print self.session['user']['uid']
print self.session["sessid"]
print self.server.file.create(file_obj)
return self.server.file.create(self.session["sessid"], file_obj)