 |
| Bunch of Cubes from Maya, imported into Blender (Color, Position) via XML and dynamically animated using Blender Bullet and rendered out. |
PREREQUISITE
- Python basics
- Python data type, especially Dictionary
- cElementTree module
- File Read and Write
PROGRAMMING AND DATA
The more one learns about programming in Python (in Blender or other 3D packages), the more one can see the pattern that the whole thing is really about "data management" and "data processing" and "data visualization":- How to retrieve data
- How to process data
- How to store data
- How to export and import data
- How to display or present the data
3D artists (like you and me) are lucky from the very beginning to work visually with computer data from the very beginning (at least in the last 20+ years). All we have to deal is the Interface. Still there is a sweet and sad labor of hours in doing what you are doing :)
However, if we really want to dig deeper with Computer Graphics, it is nice to be able to read raw data and process them in a magical kind of way.
In this post, I will be attempting to do an introduction to XML, XML Writing and Reading. I am not from programming background and writing this post is quite strange to myself as well. Anyhow, I feel like writing and sharing with you all.
IT WAS CSV
I have written a bit about CSV (Comma Separated Value) data and Blender export-import using CSV a while ago. At the time I was still clueless about Python and programming. I felt that CSV is really simple and easy to understand so that was my reason to use CSV.Sooner or later we will realize that CSV is probably limited in term of what we can "store" and "parse" from it. It does not have much of STRUCTURE apart from a long list of strings and numbers and line numbers. It can represent ROWS and COLUMN type of data, but not so much of HIERARCHY data.
See how CSV is parsed inside program like NodeBox or Houdini. It's still pretty cool:
CSV -> TABLE -> VISUALIZATION
http://blendersushi.blogspot.com.au/2012/12/python-importing-and-visualizing-data.html
\BLENDER CSV F-Curve Import
http://blenderartists.org/forum/showthread.php?209181-A-Script-to-Import-a-CSV-File-and-Create-F-Curves-(for-Blender-2-5x-or-later)
So then, what is the alternative of CSV? Just a few days ago, I learned something new (yes, new for programming newbie like me!) called XML.
I think this XML is really worth looking.
Let see what can we do with XML using Python inside Blender.
What is XML?
XML stands for eXtensible Markup Language. Apart from this long name, I think the best description and what XML really is, can be read here:http://www.w3schools.com/xml/xml_whatis.asp
http://www.w3schools.com/xml/xml_tree.asp#gsc.tab=0
The simple way I see XML is a kind of data format with tree-like hierarchy structure. XML is human readable, just like CSV, but when we look at the XML data, it is more like a Tree with Branches and Leaves.
<root>
<object1 attribute1 = "value" attribute2 = "value" attribute3 = "value">
<object2 attribute1 = "value" attribute2 = "value" attribute3 = "value">
<object3 attribute1 = "value" attribute2 = "value" attribute3 = "value">
</root>
WHAT XML CONSIST OF?
- root node, child node, grand child node, and so on. it always has root.
- tag
- element(s)
- attribute(s)
- attribute value(s)
- text
Since XML itself is a little boring to look at and actually does nothing, why do we even need to look at this? One would rather spend time doodling inside Blender already!
Be patient, my friend.
Apparently the data stored in XML structure is easily accessible via Python, just like collection of Dictionary and List. It is a lot like Library actually.
This means with XML, we can easily WRITE OUT and READ IN data. And this is a cool way we can TRANSFER DATA between any software packages or tools that supports Python. We can make them talk to one another as we can see soon.
The data inside could be anything and we can organize the data the way we like it, because with XML we can create our own custom "tags" and store the data the way we like within the XML structure.
PYTHON AND XML
Python provides many MODULE to work efficiently with XML data.http://docs.python.org/2/library/xml.etree.elementtree.html
We will be using cElementTree module instead of DOM module.
http://effbot.org/zone/element.htm
http://www.bigfatalien.com/?p=223
http://eli.thegreenplace.net/2012/03/15/processing-xml-in-python-with-elementtree/
SIMPLE VS COMPLEX XML STRUCTURE
There are simple and there are complex XML structure.A simple XML structure is when we only have 1 level of child under the root/parent.
<?xml version="1.0" ?>
<character>
<pCube1 rotateX="0.0" rotateY="0.0" rotateZ="0.0" scaleX="1.0" scaleY="1.0" scaleZ="1.0" translateX="0.0" translateY="6.36026310449" translateZ="0.0" visibility="True" />
<pCube2 rotateX="0.0" rotateY="0.0" rotateZ="0.0" scaleX="1.0" scaleY="1.0" scaleZ="1.0" translateX="0.0" translateY="12.8163721197" translateZ="0.0" visibility="True" />
<pCube3 rotateX="0.0" rotateY="0.0" rotateZ="0.0" scaleX="1.0" scaleY="1.0" scaleZ="1.0" translateX="0.0" translateY="12.8163721197" translateZ="0.0" visibility="True" />
</character>
OR
<?xml version="1.0" ?>
<character>
<MASTER rotateX="0.0" rotateY="0.0" rotateZ="0.0" scaleX="1.0" scaleY="1.0" scaleZ="1.0" translateX="0.0" translateY="0.0" translateZ="0.0" />
<simplebot_lf_knee_pv_ctrl lfLegIkCtrl="0.0" snapKnee="0.0" translateX="0.0" translateY="0.0" translateZ="0.0" />
<simplebot_rt_knee_pv_ctrl rtLegIkCtrl="0.0" snapKnee="0.0" translateX="0.0" translateY="0.0" translateZ="0.0" />
<simplebot_rt_foot_ctrl ankleStretch="0.0" fkVis="False" ikFkBlend="1.0" ikVis="False" kneeStretch="0.0" legStretch="0.0" />
<simplebot_lf_foot_ctrl ankleStretch="0.0" fkVis="False" ikFkBlend="1.0" ikVis="False" kneeStretch="0.0" legStretch="0.0" />
<COG rotateX="0.0" rotateY="0.0" rotateZ="0.0" translateX="0.0" translateY="0.0" translateZ="0.0" />
<L_FOOT rotateX="0.0" rotateY="0.0" rotateZ="0.0" translateX="0.0" translateY="0.0" translateZ="0.0" />
<simplebot_lf_heel_ik_ctrl ballTwist="0.0" footBreak="40.0" footRoll="0.0" legTwist="0.0" maxStretch="3.0" pvControl="True" stretchyLeg="0.0" toeRoll="0.0" toeTwist="0.0" />
<R_FOOT rotateX="0.0" rotateY="0.0" rotateZ="0.0" translateX="0.0" translateY="0.0" translateZ="0.0" />
<simplebot_rt_heel_ik_ctrl ballTwist="0.0" footBreak="40.0" footRoll="0.0" legTwist="0.0" maxStretch="3.0" pvControl="True" stretchyLeg="0.0" toeRoll="0.0" toeTwist="0.0" />
<REye_ctrl translateX="0.0" translateY="0.0" translateZ="0.0" />
<LEye_ctrl translateX="0.0" translateY="0.0" translateZ="0.0" />
<JG_Eye_GP eyeZ="0.0" translateX="0.0" translateY="0.0" translateZ="0.0" twist="0.0" />
</character>
They may be slightly hard to read, but still human readable and our brain can see the pattern of what it is trying to do.
A complex XML structure is when we store every values inside child of child. Here is example I took from:
http://www.w3schools.com/dom/dom_nodes.asp
<bookstore>
<book category="COOKING">
<title lang="en">Everyday Italian</title>
<author>Giada De Laurentiis</author>
<year>2005</year>
<price>30.00</price>
</book>
<book category="CHILDREN">
<title lang="en">Harry Potter</title>
<author>J K. Rowling</author>
<year>2005</year>
<price>29.99</price>
</book>
<book category="WEB">
<title lang="en">Learning XML</title>
<author>Erik T. Ray</author>
<year>2003</year>
<price>39.95</price>
</book>
</bookstore>
I reckon that the complex XML is more "correct", but then the simple XML, the single liner that can be accessed as Dictionary is sufficient for us most of the time. It is really up to us how we like to store and organize the XML.
PS: Shaun Friedberg suggested the simple XML structure. Shaun is actually the one introducing me to this XML stuff (in Maya).
CREDIT NOTE:
Originally some of the contents I talked about in here is from the Python course taught by Shaun Friedberg
http://www.linkedin.com/in/pyrokinesis
WRITING XML (EXPORT XML DATA)
Whatever value we can print() in Python, we can easily pass that as XML lines.We could type the XML lines by hand manually, but that is silly and we definitely will not do that unless we are a robot. We could of course edit the XML by hand too if we like, but that is also something we don't usually do.
We certainly use Python and LOOP to write XML.
Some module to help writing and parsing XML:
- ElementTree
- cElementTree
- dom
It will be beneficial if you search and study some Python example to write and read XML using the above modules. In our case, we will be using cElementTree, because it seems to be the fastest.
BLENDER SUSHI SCRIPT: Writing Out XML Doodle Script
'''
Preparing XML construct and Writing out XML
Based on teaching by Shaun Friedberg
Edited by Jimmy Gunawan (Blender Sushi)
'''
from xml.etree import cElementTree as ElementTree
# PRETTY PRINT
# function to pretty print the XML code
def prettyPrint(element, level=0):
'''
Printing in elementTree requires a little massaging
Function taken from elementTree site:
http://effbot.org/zone/element-lib.htm#prettyprint
'''
indent = '\n' + level * ' '
if len(element):
if not element.text or not element.text.strip():
element.text = indent + ' '
if not element.tail or not element.tail.strip():
element.tail = indent
for element in element:
prettyPrint(element, level + 1)
if not element.tail or not element.tail.strip():
element.tail = indent
else:
if level and (not element.tail or not element.tail.strip()):
element.tail = indent
return element
# Create ROOT node
rootNode = ElementTree.Element('root')
### CREATE FIRST LINE OF ELEMENT
# Create node = CHILD element
elementNode = ElementTree.Element('node')
# Parent CHILD node to ROOT
rootNode.append(elementNode)
# Adding attribute
elementNode.attrib['exampleAttribute'] = 'example value'
# Adding more attributes
elementNode.attrib['color'] = 'Blue'
elementNode.attrib['smell'] = 'Ocean'
elementNode.attrib['eat'] = 'Sushi'
# Adding text
elementNode.text = 'this is an example text'
### CREATE SECOND LINE OF ELEMENT
# Create node = CHILD element
elementNode = ElementTree.Element('node2')
# Parent CHILD node to ROOT
rootNode.append(elementNode)
# Adding attribute
elementNode.attrib['exampleAttribute'] = 'example value'
# Adding more attributes
elementNode.attrib['color'] = 'Red'
elementNode.attrib['smell'] = 'Fire'
elementNode.attrib['eat'] = 'Spice'
# Adding text
elementNode.text = 'this is second example text'
### WRITING OUT IN A CLEAN WAY IS A BIT COMPLEX... WE USE THE PRETTY PRINT
# Pretty print the rootNode
prettyPrint(element=rootNode)
# We need to convert to STRING before we can see the whole thing
xmlText = ElementTree.tostring(rootNode)
# FINAL RESULT, we need to append the HEADER
print ( '<?xml version="1.0" ?>\n{xmlText}'.format(xmlText=xmlText) )
### NOW WE LIKE TO WRITE OUT THE XML TO FILE
outputPath = r'C:\pythonInBlender\test.xml'
fileObject = open(outputPath, 'w')
fileObject.write('<?xml version="1.0" ?>\n' + xmlText.decode('utf8'))
fileObject.close()
print('XML file has been written in here: {outputPath}'.format(outputPath=outputPath))
RESULT:
Additional Reference:
READING XML (IMPORT XML DATA)
Reading data is much simpler than writing. Well, at least it seems so. Writing data seems to be more complicated at first glance.
What we need to care is what we are going to do with the data after being exported. We also need to do some Data Conversion sometimes because all the data seems to come in as String, where sometimes we need data as Integer or Float for further processing.
The data comes in as Dictionary type of data. Hopefully you have some understanding of Dictionary in Python. I am also still new to all this and it is a matter of getting the KEY and VALUE from each ITEM per ELEMENT.
BLENDER SUSHI SCRIPT: Reading In XML Doodle Script
import bpy
from xml.etree import cElementTree as ElementTree
xmlPath = 'C:/pythonInBlender/mayaOut.xml'
xmlRoot = ElementTree.parse(xmlPath).getroot()
def createMonkeyFromData():
for element in xmlRoot:
objectName = element.tag
if 'translateX' in element.keys():
#print( 'Attribute called TranslateX: {name}'.format(name=element.get('translateX')) )
locX = float(element.get('translateX'))
locY = float(element.get('translateY'))
locZ = float(element.get('translateZ'))
bpy.ops.mesh.primitive_monkey_add( location=(locX, locY, locZ) )
createMonkeyFromData()
import bpy
from xml.etree import cElementTree as ElementTree
xmlPath = 'C:/pythonInBlender/mayaOut.xml'
xmlRoot = ElementTree.parse(xmlPath).getroot()
def createMonkeyFromData():
for element in xmlRoot:
objectName = element.tag
if 'translateX' in element.keys():
#print( 'Attribute called TranslateX: {name}'.format(name=element.get('translateX')) )
locX = float(element.get('translateX'))
locY = float(element.get('translateY'))
locZ = float(element.get('translateZ'))
bpy.ops.mesh.primitive_monkey_add( location=(locX, locY, locZ) )
createMonkeyFromData()
BLENDER XML BUILDER?
There is a script called xmlbuilder.py inside Blender, I am curious what this actually does. I will check this when I have time. Or you can check it. Maybe it is something to do with DAE (Collada) in Blender.
In the mean time, I will use the WRITING XML and READING XML construct like above.
CASE STUDY: Maya to Blender via XML
Comparison of Python in Maya and Blender
I have to admit that Maya provides a lot more (shortcut) commands that allow users to quickly get ATTRIBUTE or VALUE data from one object (or node in Maya).
A little bit information:
At the current state, Python in Maya is well-known as "Wrapper" of MEL (Maya Embedded Language) programming. MEL has been there in Maya for years, before later Python came in.
MEL scripts itself has hundreds of handy scripts to access all kind of data from nodes. But, even so, Python handles "Strings" and "List" like in a really awesome ways, and this makes writing script is surprisingly easy and data flow is becoming a breeze in Maya.
It will be a little weird for people learning Python in Maya because they have to always look at the MEL first. They need to kind of learn MEL too, at least to really understand the functions and reading the history.
There is PyMEL from Luma Pictures that allow for more Pythonic way to access data, kind of the more correct way Python should implemented in Maya. This is more like how Python in Blender works. It is all Object Oriented (OOP).
There are some Python API that allows to access some Maya's core that normally only accessible using C++ programming. But it is pretty complicated and low level and I am not familiar with those yet.
I am still new to Python in Blender and Blender BPY stuff and still getting familiar with the construct. I am definitely not 100% sure with a lot of things BPY. Meaning, I have a lot of questions regarding the API itself.
Like I said earlier, it seems to be much easier to access DATA inside Maya.
Examples of some handy "commands" in Maya:
- List
- Select
- Get Attribute
- Set Attribute
- Connect Attribute
- List Attribute
- xform
Blender Python "does not" come with all those kind of commands. BPY feels more "raw" in my opinion.
Users may actually write their own COMMANDS that can mimic all those handy tools that are given from Maya commands.
In Blender, you are accessing the data and value more directly. In Maya, they give some commands to access the data and value in more handy kind of way.
Exporting data From Maya To Blender using XML
Let's take a look at example that I tested the other day.
I am inside Maya and I have 3000 cubes created using Python in Maya, which was based on 3000 Particle POSITION and COLOR. The Particle itself inherited its Colors from Procedural RAMP texture.
You can read a bit more at my Maya blog:
http://mayaspiral.blogspot.com.au/2013/04/python-getting-particle-rgbpp-and-apply.html
In Maya, it is super easy to query ANY Attributes and Values of Particles, including custom attributes per Particle that user created.
In Blender, we can also easily get Particle Attributes such Position, Size, etc. but still pretty limited at the current state. Blender unfortunately cannot access Particle Color by its index. Not at the moment, but I think pretty soon in the future. Fingers crossed.
EXAMPLE: Blender Particle, get position, create cube at position
For example, if I want to create some bunch of cubes at Particle Position, we could so something like this:
import bpy
obj = bpy.context.active_object
ps = obj.particle_systems.active
#ps.particles[0].location
def printPosition():
for i in range(len(ps.particles)):
location = ps.particles[i].location
print(i, location)
# Create Cube At Position
bpy.ops.mesh.primitive_cube_add(location=(location[0], location[1], location[2]))
selectedObject = bpy.context.selected_objects
selectedObject[0].name = 'pixelcube.%s' % i
selectedObject[0].scale = 0.45, 0.45, 0.45
Now, the reasons I am doing the XML data export from Maya to Blender are below:
- Because I want to get more familiar with Python and XML data export import
- Because Maya's Bullet is currently sucks. It is very limited and slow.
- Because Blender's Bullet is awesome and fast.
- Because I am thinking of Pipeline where Blender can be the Renderer, Compositor, etc. And I like the practical Blender Fire and Smoke and some more VFX things.
And so I wrote script that does the exporting of some specifc datas from some nodes (every objects in Maya is nodes).
Basically from the 3000 cubes, I exported the COLOR RGB information. That is what I need. I could export the POSITION data as well, but for that I actually just use DAE Collada. Same thing.
How example XML data from Maya looks like:
 |
| Simple Tree View in Chrome |
 |
| The actual XML Data |
Importing data Into Blender From Maya using XML
As the data is now "outside" of Maya as XML, we can do anything with it.
I bring the 3000 cubes from Maya as Collada DAE, Blender import them fine with the node names in tact and including Material that was set in Maya. Make sure the Unit Size is checked.
I wrote the script below to READ the XML data.
XML, Dictionary, Strings
XML, when imported will come as Dictionary type of data. And it is also STRING type of data. I imported the RGB like this "[0.124, 0.55, 0.123]" and they are strings. Where the data expected supposed to be 3 values of Float.
In my script below, I have noob attempt to convert 3 values string into 3 values float.
BLENDER SUSHI SCRIPT: Importing RGB via XML (but you need to setup the XML)
import bpy
# import xml.etree.ElementTree as ET
from xml.etree import cElementTree as ET
def applyColor(object='pCube999', rgb = [1.0,0.0,0.0]):
# Get specific object
object = bpy.data.objects[object]
# Create new vertex color
object.data.vertex_colors.new()
# Get Total Length of Vertex Colors for each selected meshes
totalVertCol = len(object.data.vertex_colors[0].data)
# Iterate over every mesh vertex color and give it a single colour
for i in range(totalVertCol):
object.data.vertex_colors[0].data[i].color = rgb
filepath = 'C://xml//test.xml'
tree = ET.parse(filepath)
root = tree.getroot()
print(root.tag)
# root = ET.fromstring(country_data_as_string)
for child in root:
# Get node name
nodeName = child.tag
# Get Dictionary (including key and value)
dict = child.attrib
# Extract value from Dictionary
rgb = dict.get('rgbVal')
rgbClean = rgb.strip( '[]' )
rgbList = rgbClean.split(',')
rgbColor = float(rgbList[0]), float(rgbList[1]), float(rgbList[2])
print (rgbColor)
applyColor(object='{nodeName}'.format(nodeName=nodeName), rgb = rgbColor)
Thanks again to Taisuke Tanimura and his magic of list comprehension, there is a better way of doing that 3 values String RGB into correct 3 values FLOAT:
rgbList = [float(token) for token in rgbVal[1:-1].split(',')]
Big thanks to:
- Shaun Friedberg
http://workshops.cgsociety.org/instructor.php?userid=14448
http://pyrokinesis.cgsociety.org/about/
http://www.imdb.com/name/nm3571084/ - Taisuke Tanimura
http://www.imdb.com/name/nm2552359/
http://somewhatslanted.com/yoga/
Post a Comment