I'm excited that Sivaji (sivaji) has been selected to work on the Quiz module for Google Summer of Code and by the recent improvements that Matt (mbutcher) has been making.
I'm writing to propose a kind of big idea that could give structure to the next 3 months of development. Right now the Quiz module handles authoring quizzes, searching for items, and taking the quiz. What if the first two functions were broken out into a Quiz Bank or Assessment Item Bank module separate from the learner Quiz view? I think this would be very useful to educators who want to discuss quiz items and use them in their exams or homeworks. They aren't all concerned with running a quiz in Drupal because they have some other LMS such as Moodle or Blackboard at their school. Instead, the Bank module would let them create and find the best items that they want to use and then export them to their LMS. Of course, that export could also go to the Quiz (Taking) module.
With this perspective, modules could be written for other LMSes to draw on Bank content in Drupal. For example, the Question Bank module in Moodle has an API to make a Moodle instance into a hub that others can draw on for items. We could implement that API in Drupal's Bank to let Drupal serve as that hub, which would be much better as Drupal has more features to support community interaction. Even better, the same question items would be available in other LMSes such as ATutor, SAKAI, ILIAS, etc.
What do people think of this idea? I'm interested in helping support the hypothetical teacher community I described as part of my PhD research in human computer interaction. If there's support from the community and module maintainers, I will put in the coding time to implement it.
cheers,
Turadg
Comments
Moodle import & export formats
Hi Turadg,
I think this is a great suggestion and a step forward to better interoperability between Drupal and Moodle.
Moodle provides some formats for importing and exporting questions (and other things). A good starting point is looking at http://docs.moodle.org/en/Import_and_export_FAQ
Cheers,
Frank
Frank
My LinkedIn profile
Moodle integration use cases
hi Frank,
I'm glad other Drupal users are concerned with interoperability with Moodle. Do you have any other use cases for how they can connect?
I've delved into the Moodle import and export formats to try re-using their code to allow the Quiz module to import from and export to the same formats, but their code is tied pretty closely with the Moodle database model. It would be nice if the internal models of both Drupal Quiz and Moodle Question Bank used the QTI data model, but I know that model can be quite complex.
Who has a use for using Drupal content (such as quiz items) in their LMS (such as Moodle)? I'd like to hear your experiences and ideal scenarios to help guide the design.
http://educoder.org/
http://openeducationresearch.org/
Data and a use case
RE: "but their code is tied pretty closely with the Moodle database model" --
This is the main barrier to exporting quiz/lesson content from Moodle into other apps -- within Moodle, the data is both tied to their db model, and a specific set of interactions/rules about how users interact with that content. So, it's more than an issue of straight import/export -- it's also a question of what functionality to replicate between the two systems.
We have just done some work moving resources between Drupal and Moodle, and one of the essential design decisions was what data to include, and what to exclude. At some point, you reach a point of diminishing returns, where it's better to let Moodle be Moodle and to let Drupal be Drupal. In short, identify what makes sense to be native Drupal, what makes sense to be native Moodle, and what is common functionality. It's that middle area where you have the best options for transporting content between the two apps.
This is also where the ongoing Drupal/Moodle integration work (at http://drupal.org/project/moodle ) merits some close examination. We haven't taken this for a trial run yet, but the approach looks spot-on -- I particularly love the fact that they built out a RESTful API (at http://code.google.com/p/qapi/ ).
Cheers,
Bill
FunnyMonkey
Click. Connect. Learn.
Using Drupal in Education
FunnyMonkey
QTI format & Moodle
Here you find all the discussions on QTI format in the Moodle forums http://moodle.org/mod/forum/search.php?notwords=Re:&id=5&subject=qti
There's some recent discussion on "Develop QTI 2.0 import" (http://moodle.org/mod/forum/discuss.php?d=118025) and "Quiz - IMS QTI export" (http://moodle.org/mod/forum/discuss.php?d=118507).
Cheers,
Frank
Frank
My LinkedIn profile
Drupal-Moodle integration
Hi turadg,
most efforts I know of regard Single Sign-on (SSO) for Drupal and Moodle.
Cheers,
Frank
Frank
My LinkedIn profile
Deskription of the moodle-Questions Data-Format ("GIFT")
... i think e better deskription of the moodle-Questions Data-Format ("GIFT") in to be found on the moodle-documentation page:
http://docs.moodle.org/en/GIFT
...
maybe it would be useful, not to re-invent the wheel ... and use a data-format, which has probably some advantages:
a lot of Question format examples
existing (particulary) open-source applications, which ara already using this format ...
... for example:
a template for ms word (a) / open office (b) for editing large amounts of questions ... (very useful!)
simple utf-8- format!!!
Greetings from an enthusiastic moodler (all what has to do with elearning, course-design, didactic formats) with great symparthy for drupal (for the fantastic community & collaboration -options ...)
Heiko Idensen
P.S. a nicw way to test different Quiz-format is
Charles Kelly's Online Quiz Generator (c)
A Reference for the GIFT-DATA_Format (d)
Links:
a) http://www.soberit.hut.fi/sprg/resources/moodle/GiftConverter.html
b) http://moodle.org/mod/forum/discuss.php?d=20705&parent=168385 (you have to subscribe before, sorry!)
c) http://a4esl.org/c/qw.html
d) http://buypct.com/gift_reference.pdf
GIFT vs Moodle XML format
Welcome Heiko. Thanks for bringing your Moodle experience to this thread.
I also support Drupal importing GIFT format. I don't think it's worth exporting to GIFT though. Here's why.
GIFT is basically an authoring format, not an interchange format. It makes it easy for people to write questions in a text editor. Consequently, it can be very difficult to parse and to my knowledge the only reliable implementation is in Moodle and that nothing interoperates with it well because the parsing is so irregular. Moodlers have said [1], "The format of GIFT makes it difficult to add new features without making it very error prone. We are almost at the point where any text parses as something. If you need comprehensive import facilities then you should look at the Moodle XML format."
So to import into Moodle—exporting from something else—it's best to use a more well-specified and extensible format. QTI would be ideal since it has an even larger community around it, but it is also more complex and currently Moodle doesn't support importing QTI very well (see elsewhere in this thread). So I believe that the best way to support sharing questions items with Moodle is to export in Moodle XML format.
-TA
[1] moodle.org/mod/forum/discuss.php?d=122768 (I had to kill the url to get past g.d.o's faulty spam filter)
http://educoder.org/
http://openeducationresearch.org/
My comments are not a direct
My comments are not a direct response to this topic but are related. As a classroom teacher I wish there is a central question bank online from where I can select and then, maybe covert into some sort of code first, import selected questions to my Drupal site for use. Textbooks usually come with test bank questions. why not make a question bank website to feed questions to moodle, drupal, and others. If I want to customize some questions then I will be able to do so, too. Thanks.
Open question bank, yes!
kding, thanks for your comment. What you describe is exactly the project I'm working on as part of my research. If you have some time for an IM chat or voice call, please get in touch with me at turadg(at)cmu.edu.
http://educoder.org/
http://openeducationresearch.org/
Quiz/test/educational markup language
kding, turadg
I remember when years ago, with XML formats popping up everywhere, there was an initiative to develop an Extensible Educational Markup Language that could serve as a standard data format for exchange between electronic learning solutions. I did not hear much about it since then, but something along those lines, as expressed by kding above, would be a perfect foundation for a universal test repository for learning management systems. Maybe a module for Drupal and for Moodle would be able to connect to it and immediately implement it in a course. Each system's own module would create a database table in the same format the other platform could directly read from. Almost like synchronized browser bookmarks.
This just further emphasizes the importance of founding a front for the DrupalEd concept, a body/alliance/association that could also advocate the development and implementation of open educational data exchange formats. The Drupal and Moodle communities have always enjoyed mutual respect for each other, recognizing the fitness of the two systems for specific purposes. With the proliferation of open source solutions in education (no doubt partially spurred by the economy lately), this would be a perfect time to step up communications between the various groups. A solid community, represented by an advocacy group, would give a tremendous amount of credibility to the platforms, would quell the FUD, fill educational institutions with confidence to take the open source plunge, and would also steer more traffic to a myriad small businesses that provide service for the various systems.
Ever being the Drupal "activist", here is my writeup on the subject at another forum:
http://groups.drupal.org/node/11433#comment-74214
===================
http://www.elteto.net
===================
http://www.elteto.net
exchange formats
Elteto, a general EML has been the holy grail among some edtech researchers and developers. It's quite hard. Mostly because it's hard to agree on how education works. Specs like Learning Design try to encompass a wide array of pedagogical practices, but then aren't implementable. Specs like SCORM describe learning activities, but in a very limited way.
I definitely agree that exchange formats are the way to go. I think the question bank is the best to start with. While a question definition can get complicated, it is a tightly defined artifact. There are simpler artifacts but so simple that you can just copy and paste. I hope we can develop this question bank and then we can grow out from there.
http://educoder.org/
http://openeducationresearch.org/
Standardized question types, statistics
turadg,
I know what you mean about various specs. Here is an example of the system guidelines I had to use when working with one educational agency. 429 pages of philosophy, specifications and procedures. However, it did have a defined structure, and it was closely tied in with student administration as well.
http://www.elteto.net/sites/elteto.net/files/sat_manual.pdf
I know your time is valuable, so just peek into Page 62 (2-34) to see some of the question format specifications.
Certainly, question types must be established, such as essay, multiple choice, fill in the blank (with adjustable levels of input tolerance), essay (requires scoring input AFTER testing), single choice, true/false, mix and match, etc...
Your use of the term "Interactive Assessment" is right on, and it reminds me of using Macromedia Authorware. You could design your own interactive tests by selecting the type of test question, add multimedia, and designate the point value of each correct/incorrect answer. And, this is where exchange format comes in, it stored its data in a .mdb format, so I was able to tap into it for statistics and even design a web application for it in ColdFusion (yes, that was before the Drupal revolution...). Of course, it had a proprietary data definition, but it was easily decipherable.
Having worked as a curriculum developer, I am curious to see how you approach quiz/question definitions from the pedagogical angle as well. Primarily, are you proposing the adoption of an existing data exchange format, or should there be a unified push for a new one? I think the quiz concept should be separated into three elements: question bank, assessment design, and statistics/reports. This would allow educators to develop their own scoring system independent from the question bank. A separate assessment design module would give quiz authors the choice to decide in what format to present questions in.
I am definitely up for some ed-chat on the subject as well.
===================
http://www.elteto.net
===================
http://www.elteto.net
Use existing specs
Elteto,
Thanks for the link to that manual. I think those pages have some useful principles to guide authors of questions. Maybe some of that could go into a help system for the question authoring tool.
The question types in there are pretty standard. So fortunately they're already covered in existing standards. For better or for worse, there are actually many "standards" to choose from. Just look at this list of formats Moodle imports. And that doesn't even include the biggest standard of them all, QTI.
So I don't think we should come up with any new specifications. Let's pick one or two to support very well. I think QTI and Moodle XML would get the most traction. Though I would propose importing from GIFT and Aiken formats since they're human-writable notations for authoring a bunch of questions quickly.
I agree that eventually we may want to have three different modules: Question Bank, Interactive Assessment, and Assessment Reporting. I only have the drive to pull out the Question Bank module, so I'm personally content to see reporting stay fused to the assessment code in the short term. (That would allow more flexibility in the data model for recording responses.)
http://educoder.org/
http://openeducationresearch.org/
YES! This is where we want to go!
turadg: This thread made my day! Thanks for starting it.
There are several things to talk about... so I apologize for this long post.
Quiz/Question Bank: I would love to see this happen. A pure question/quiz management module just used for composing and managing would be great. I think even the ability to generate good old fashioned paper-based quizzes from this would be highly desirable.
Interactive Assessment module: I would envision wrapping up all of the actual learning assessment logic into one or more modules. In this framework, we would have different methods of administering an online exam. It might look something like this:
Ideally, I'd like to move personality quizzes into their own separate module, since the logic for those is so different that it requires hacks all over Quiz to accommodate.
QTI Data Exchange: Here I am feeling a little sheepish. See, I've been working on a QTI importer not realizing that QTI is a standard format (I thought it was just for QuestionMark Perception). Using the QueryPath module, I can very easily convert Drupal structures into XML (and vice versa).
Now that I realize the potential for having QTI XML, I can envision adding a gateway that would allow administrators to export quiz questions as QTI files. If Moodle and others can take those files and make use of them, then we have a clear winner for data exchange.
What has to happen for any of this to actually see the light of day
As great as these ideas all are, there is a whole lot of work involved in making them happen.
The QTI stuff looks like it is going to be done. I have a project that will use it.
Splitting the module up into smaller parts, though, is a huge project. Sivaji already has a clear agenda for his GSOC work, and my time on Quiz is pretty limited (and will soon become more so, as my main Quiz-centered project is about done). So... if any coders want to help... it might be a good Quiz 4 project.
Matt
Blog: http://technosophos.com
QueryPath: http://querypath.org
Blog: http://technosophos.com
QueryPath: http://querypath.org
Interactive Assessment
I think "Interactive Assessment" is a great name of the module we're talking about. It can take care of the interaction logic and evolve separately from the bank, with a clear question data model linking them.
I agree that personality quizzes should get a separate module as their logic is different.
All the stats, views, reporting and notification would fit into this module.
http://educoder.org/
http://openeducationresearch.org/
Hi, this is a great
Hi, this is a great direction.
I would like to add that quiz statistics would be great to have with the quiz-submissions. Statistics related to averages, or by a specific usergroup, etc.. Great development here!
Greetings,
Martijn
Statistics and Quiz 3
Martijn,
sdboyer and I have been working on some pretty heavy-duty views integration. At least some statistics can now be generated out of custom quiz views. There might even be some examples in the dev release (I can't remember).
Another option (now that we have full Views support) is to export custom views reports into CSV files (see View Bonus Pack).
We should talk about statistics, though. A list of statistics that would be beneficial is probably a good place to start.
Matt
Blog: http://technosophos.com
QueryPath: http://querypath.org
Blog: http://technosophos.com
QueryPath: http://querypath.org
hi Matt, Great quiz has full
hi Matt,
Great quiz has full views2 support!
Statistics I am thinking of are:
-off course immediately after taking quiz, how the quiz has been scored. In table and may be graph.
To be able to do this, there must be the possibility to score every fill_in_the_blank, and multiple choice answers (all sorts, one good, more answers good, etc..)
- then the average quiz-result in total against all people who did the same specific quiz, may be also possible to use taxonomy to tag a quiz on a certain subject, and then only count the results of this specific quiz about this specific subject (taxonomy)
- then the total average quiz results as possibility so how the quiz has been scored all time.
- end the recent average quiz results.
So possible a block and page possibility to redirect to immediately after the quiz has been done with the above statistics possible in one table, and in view-fields?
Greetings, Martijn
Exporting to Moodle
QTI
QTI is indeed used by more than Respondus. It's a spec by the IMS Global Learning Consortium. It's a pretty daunting spec because it tries to model all types of questions and thus it makes some of the simplest cases harder than they need to be. (e.g. standard multiple choice questions) It's easy to export to QTI because you just make a QTI template for your interaction type and fill it in. But importing QTI is much harder as you have to accommodate the general model, or at least carve out a subset. So Moodle does not import QTI generally . There is a module that will import the Respondus subspec of QTI. So maybe mbutcher's work in exporting to the Respondus format is best for now.
Moodle XML
We can imagine lots of integration points with Moodle. SSO seems to be handled pretty well. I'd like to eventually see using that in Moodle quiz authoring to import from a Drupal question bank module. One essential step to that is exporting questions to Moodle. The Respondus export (above) would help, but it might be easier to use the Moodle XML format. It would work with all Moodle instances (the Respondus importer is an add-on) and would be less convoluted than going through a proprietary subset of an unwieldy giant spec.
http://educoder.org/
http://openeducationresearch.org/
QTI 1.2 work is underway
I began "official" work on QTI 1.2 on Friday. I will be building an import/export library that integrates with Sivaji's quiz_import module.
My reference platform is likely going to be questions from/to QuestionMark Perception. If you would like to send me some of your sample data, too, I would be happy to test with that as well.
Matt
Blog: http://technosophos.com
QueryPath: http://querypath.org
Blog: http://technosophos.com
QueryPath: http://querypath.org
Next steps to making this happen
We have the overall vision now. And it's validated by at least one educator on the thread. :) I think next is the technical vision, a roadmap of how it will come to be. This project needs use a high level plan for separating Quiz into two modules and a list of tasks to get there.
I'm signing up to lead this and do much of the coding. Of course, since I'm new to Drupal and this development community, I will be much more productive with guidance. mbutcher, your time is limited and getting more so. Will you be able to lay out a plan to help coordinate volunteers? Sivaji, reading your GSOC wiki it looks like you're focused on this "Interactive Assessment" functionality, which right now lives in the Quiz module. Would you be interested in helping separate out the author parts from Quiz to let it focus on these interaction and reporting parts?
Anyone else here who might provide design input or even coding?
Up for a meeting on #quiz-drupal?
http://educoder.org/
http://openeducationresearch.org/
XML considerations
XSLT
I would consider doing the XML import/export with XSLT as an intermediary step. This would make for a looser coupling between Drupal and other LMS (e.g. Moodle).
XML Schema for Moodle XML
I have been planning to create an XML Schema for Moodle XML (but haven't finished it yet due to other tasks).
OpenLearn XML
Another option might be using OpenLearn XML, used by The Open University (http://www.derby.ac.uk/pocket/cdk/xml http://openlearn.open.ac.uk/local/sa/schemas/OUGeneric_v1.2.xsd).
JFTR: I pointed to this discussion in the following Moodle forums:
http://moodle.org/mod/forum/discuss.php?d=121859
http://moodle.org/mod/forum/discuss.php?d=119857
Cheers,
Frank
Frank
My LinkedIn profile
XSLT
In theory, I like the idea of using XSLT. In practice, it is complicated when each XML format actually captures different data.
For XSLT to work, we would need a canonical XML format (one that had all of the necessary data). From there, somebody (and I am NOT volunteering) could write XSLT stylesheets to transform from the canonical format to any desired target format (such as OpenLearn, QTI, or Moodle XML).
Again, I will be working directly on QTI 1.2 import/export (with QuestionMark Perception as the reference impl) because I have an immediate need for it. Might it be possible to transform from QTI to Moodle XML using an XSLT?
Otherwise, the moodle format is actually simple (compared to QTI). That is something I'd be happy to work on, as well.
Matt
Blog: http://technosophos.com
QueryPath: http://querypath.org
Blog: http://technosophos.com
QueryPath: http://querypath.org
@turadg
I am excited to see one more guy taking initiative to improve quiz module. Sure i am ready to help you in all possible ways. I am a newbie haven't worked with moodle or other LMS (except Drupal) but i have great passion to learn them.
My primary focus will be on features listed in GSoC wiki http://groups.drupal.org/node/21627 (will be updated soon), and http://groups.drupal.org/node/19141.
<?
I like to start working with Import/Export feature(2.2.1 => #2) of my SoC proposal [1] already i have submitted a patch [2] which allows to import quiz questions (multichoice, true/false) in CSV format. This week i will be working on Aiken format [3]. I think this Aiken format is more simpler and relevant to YAML which mbutcher suggested me to refer. i think this format will ease the interoperability between Drupal and Moodle. Probably this weekend i will try to finish it with both Import and Export support. Hope you guys will help me to test and review it.
[1] http://sivaji.ubuntu-tam.org/soc2009_application.pdf
(sorry for posting pdf i will update by GSoC wiki page ASAP)
[2] http://drupal.org/node/210408#comment-1502854
[3] http://docs.moodle.org/en/Aiken
--
sivaji
ubuntuslave.blogspot.com
Aiken versus GIFT
Sivaji,
I'm glad our IRC chat was helpful. Since I proposed both Aiken and GIFT[1], I'm interested in how you decided on one. My intuition is that supporting GIFT would do more to reach out to Moodle. What led you to you choose Aiken?
-t
[1] http://docs.moodle.org/en/GIFT
p.s. anyone know why the spam filter is so sensitive? Whenever I use hypertext the filter blocks me. (So this time the footnote notation.)
http://educoder.org/
http://openeducationresearch.org/
The following reasons made
The following reasons made me to start with Aiken
--
sivaji
ubuntuslave.blogspot.com
QTI Importer
Sivaji,
I had to whip up a QTI importer yesterday, so I have a basic implementation. It's about 150 lines of code, and uses QueryPath to parse the QTI XML. I wouldn't say that it is terribly sophisticated, but it does a fair job of matching QTI 1.2 data to Quiz question types.
It is not checked into CVS. I'd rather give it to you first and see if you would like to integrate it into question_import.module.
Matt
Blog: http://technosophos.com
QueryPath: http://querypath.org
Blog: http://technosophos.com
QueryPath: http://querypath.org
Moving Forward
This thread continues to be a successful discussion on where Quiz should go, and we do have several people interested in development.
On my part, here's what I am going to do. I am going to begin the process of releasing Quiz 3. I will cut an Alpha or Beta release by the end of the week, and we can work on stablizing that.
From there, let's begin work on Quiz 4. The remainder of sivaji's GSOC items will all go in there. (He has already made several contributions which will go into Quiz 3). Turadg has offered many ideas, as well as development support. I'd love to see his ideas realized in code.
So... are we ready to start a serious discussion about how features mentioned here can be distilled into a roadmap for Quiz 4?
Blog: http://technosophos.com
QueryPath: http://querypath.org
Blog: http://technosophos.com
QueryPath: http://querypath.org
Make it a separate project?
Shall we continue using this thread? Or better make it a separate project?
What I would like to have is a repository of sample files for the different formats for development, testing and evaluation purposes.
Cheers,
Frank
Frank
My LinkedIn profile
New thread?
We should probably start a new thread for Quiz 4.
For sample files, we probably would need a repository somewhere.
Blog: http://technosophos.com
QueryPath: http://querypath.org
Blog: http://technosophos.com
QueryPath: http://querypath.org
@mbutcher sure mbutcher send
@mbutcher
sure mbutcher send me the code i will port it to questions_import.module. Thanks for taking initiative to discuss about Quiz 4 GSoC project.
@turadg
Now quiz 3.x has Moodle Aiken format questions import feature.
sivaji
ubuntuslave.blogspot.com
separate thread
Make it a separate thread, the conversation is going beyond limits.
http://jobduties.org/
Test Bank
Anyone can help me? Actually I want test bank for instant use so please tell me that this place is secure or not?
http://www.testbank-solutionsmanual.com
Thank You in Advance!