I don't know if this is configuration issue, or a limitation of Panels2, but here on g.d.o, if you
1) Have two groups, each with different vocabularies.
2) create a node in Group X, you see the categories options associated with that group, and can happily set them.
3) Use the checkboxes to assign the node to Group Y.
4) Browse over to Group Y, and edit that content.
5) Now you set the category options associated with Group Y (which is fine), and resubmit the node.
6) And finally, when you travel back to the original Group X, look at the node again, it's original category info has been lost. The Group X category values have been reset to their default state.
This is the first real PITA I have run into running BDUG as a virtual subgroup of Bay Area group.