import products/users from XML/CSV files

We encourage users to post events happening in the community to the community events group on https://www.drupal.org.
erlendstromsvik's picture

We are working on several online shops using Drupal + the e-commerce package. At the moment we are looking into integration with backend systems and datafeeds. Using either Drupal modules or programs which will interact directly with the drupal/e-commerce database.

Some stores use their own backend systems and they would like to regularly update their online store using xml/csv-files. This is a major feature for some of the companies we develop for. Some of our in-house made online shops have 100'-200' product updates each day. Updates which come in the form of large automated xml-feeds.
Although the largest sites are not going to move to Drupal yet, there are still several medium sized companies, with physical stores and their own backend systems, who want to be able to automatically create/update products on their online catalogues/shops by xml/csv datafeed.

I've found several modules for importing of nodes into Drupal and one for users. Changing one of these to work with the e-commerce package, shouldn't be too hard. Although I would like to know if such a modul already exists.

At the moment we are creating python programs to directly work with the Drupal/E-commerce database for importing/updating products/users. It would be handy to later have modules to such work on a smaller scale, so administrators of small sites could do such work "by hand".

Are there any modules/scripts for bulk import of products/users already made for the e-commece package?

Comments

Import / export API

Jaza's picture

I don't know of a solution for importing / exporting e-commerce data at the moment, but the user import module might be able to help you a bit with users, and the node import module should help with importing nodes in general (you might be able to extend it to support ecommerce data).

Thinking longer-term, I'm developing an import / export API module for Drupal, for the Google Summer of Code. I would be interested in knowing the format of the XML and CSV data that you need to import and export, to ensure that the API I'm developing can handle needs such as yours.

Jeremy Epstein - GreenAsh

Jeremy Epstein - GreenAsh

Import scripts

erlendstromsvik's picture

Right now I'm creating scripts (in python) to load products/terms (terms as product categories) and users/adresses into the necessary tables. The scripts will create new "items" if they don't come with an ID or if the supplied ID is not found in the database, or, if an item ID exists, update that item with the new information. Will probably be done with the scripts, this coming weekend.

I choose to go "outside" Drupal and it's modules right now, because I need this functionality ASAP. It does however give me quite a bit of insight into the Drupal and E-commerce database structure.
I'll come back with some more information about how we structure the CSV/XML-files later. Have to get everything working first and tested on a few sites.

That import/export API module sounds like a very much needed update for Drupal! I'm really looking forward to see how it works out.
Went through the code of the user import module, to get a few tips on how to write the user/adress-script.

Erlend Stromsvik - erlend@nymedia.no

Erlend Stromsvik - erlend@nymedia.no - erlendstromsvik @twitter
Ny Media AS - http://www.nymedia.no

Import/Export API will be a

sun's picture

Import/Export API will be a perfect solution for this issue.
I didn't look at the specs yet, but I doubt that one will be able to Import categories from other Systems by CSV/XML. Items are unique, even more unique with SKUs (Drupal node IDs don't help at first imports). But Drupal taxonomy is a system you won't find in any other shop system. You only have one categorization for your products usually. However, some systems provide functions for "linking" an item to other categories.

jaza, I think it would be a dream if your Import/Export API would support pluggable extensions, each providing settings on how to import data fields into Drupal. There could be a osCommerce/openCommerce plugin, one for xtCommerce and all the others.

Daniel F. Kudwien
unleashed mind

Daniel F. Kudwien
netzstrategen

Agree

poopertropper's picture

"jaza, I think it would be a dream if your Import/Export API would support pluggable extensions, each providing settings on how to import data fields into Drupal. There could be a osCommerce/openCommerce plugin, one for xtCommerce and all the others."

I think that is really what would be the best.

Already possible

Jaza's picture

The API already allows module authors to write their own data definitions to match a particular database schema. Anyone who's interested can go off right now, and write a set of definitions to make the API compatible with a database system completely external to Drupal, such as osCommerce. To actually use the API with a database separate to Drupal, you'd have to open another database connection in your code - but apart from that, it shouldn't really be any different to using it within Drupal.

I'd be very interested to hear about people's experiences with using the API in this way.

Jeremy Epstein - GreenAsh

Jeremy Epstein - GreenAsh

Databases?!

sun's picture

BANG! What? Database Import? That sounds awesome!
I'm still thinking about importing a whole Joomla site for migration to Drupal. To make this possible Import API would have to support the static assignment and transformation of IDs - use case: Import all users (getting new Drupal user IDs) and after that import all forum posts (at which user IDs have to be mapped/converted to the new user IDs). Do those implemented references support that?

Awesome work Jaza!

Daniel F. Kudwien
unleashed mind

Daniel F. Kudwien
netzstrategen

Joomla to Drupal

Anonymous's picture

I need to set up a shopping cart site ASAP, for a client who is retailing & wholesaling clothes. Joomla looks nice & easy. My Guru claims Drupal to be functionally richer, but there do not appear to be many instances and it seems I would be breaking new ground. We want to use the stock database of a locally hosted accounting system because there will be a lot of updates and the client will also need stock control and later POS.
Any guidance you offer will be invaluable at this stage.
Regards, Allan

Depends on requirements

sun's picture

This can't be told in general. Drupal is a beast, but E-Commerce lacks in common store functionality.

Daniel F. Kudwien
unleashed mind

Daniel F. Kudwien
netzstrategen

A must

poopertropper's picture

This is the kind of functionality that a lot of other e-commerce sites already have. In fact, it the main reason that I haven't moved all my clients over to drupal as an ecommerce solution. To many of them use there own backend, or if they needed to switch, do not want to enter in all their data over again. Even with as little as 80 products this can be tiresome. I would love to hear how you use your current python script as I have no solution at this point.

Bestpractises XML import and Ecommerce solutions?

Summitnl's picture

Hi,

I am looking for bestpractises of XML-imports and Drupal Ecommerce solutions like the ecommerce and the ubercart solutions..
Does anyone already import XML from a wholesaler to his E-commerce shop? If so, with what tooling?
Is there a module or tweaking "in the make" for this. Does the importexportapi or feedapi module be of use with this?
Please advice how to proceed with this.

My idea is to be able to import an xml-file from a wholesaler. Then be able to make product-nodes from this xml-file and when a sale is made the handling should also be done within drupal.
Some bestpractises around wo want to share there views or solutions for this? I am available through email also.
Thanks a lot in advance for your reply!

Greetings,
Martijn

Excel/CSV Import

FreeGroup's picture

Hi,

I know this is a vey old thread.....but still interesting for other users.
We use www.dbTube.org to import users into our database.

Greetings

Andreas