Tutorial Art Design: export

Articles by "export"

SVERCHOK / When Python Plays With Cricket

I am currently re-reading "Generative Algorithms" PDF book for Rhino-Grasshopper learning. It is an excellent read while learning the Sverchok (cricket) sings in Blender. You MUST read this ebook if you are serious about procedural/parametric modeling.

In Sverchok, from the very beginning, we are already exposed to DATA and DATA STRUCTURE. The DATA in Sverchok is simple and seems to be limited for now. Limitation is actually good in this case.

The DATA STRUCTURE in Sverchok is actually quite interesting, in fact Thinking about Data Structure is another advance step to take you to the next level, whether in programming or in procedural workflow. We will be able to modify, convert, re-arrange DATA more correctly if we understand DATA STRUCTURE well.

I do not have programming background, but recently encounter this "data structure" term quite often. I am probably understanding this "not quite right", however this is how I understand it now.

All these are very easy for programmer, not so easy for non-programmer. I am writing from non-programmer perspective.

DATA

In Sverchok, you will encounter data which are mostly numbers.

FLOAT = fractional values, eg. 0.5256, 0.2
INT (integer) = whole numbers, eg. -4, 0, 135

DATA STRUCTURE

Tree Structure
Tuple
List
List within List
List within List within List

I think the analogy of Russian Matryoshka doll is perfect and brilliant to explain this. I am still slowly trying to understand the whole article here at Sverchok main website:

http://nikitron.cc.ua/Sverchok_man_02.html

I think at the very basic, we can think of DATA STRUCTURE like TREE with BRANCHES.

In Sverchok, we have this interesting simplification of DATA and STRUCTURE. When we bring a simple object mesh, like Blender default Cube object:

Using Sverchok VIEWER TEXT node, we can see the actual DATA and STRUCTURE of Poly Cube.

vertices:

(1) object(s)

=0= (8)

(1.0, 0.9999999403953552, -1.0)

(1.0, -1.0, -1.0)

(-1.0000001192092896, -0.9999998211860657, -1.0)

(-0.9999996423721313, 1.0000003576278687, -1.0)

(1.0000004768371582, 0.999999463558197, 1.0)

(0.9999993443489075, -1.0000005960464478, 1.0)

(-1.0000003576278687, -0.9999996423721313, 1.0)

(-0.9999999403953552, 1.0, 1.0)

data:(1) object(s)

=0= (12)

(0, 1)

(0, 3)

(0, 4)

(1, 2)

(1, 5)

(2, 3)

(2, 6)

(3, 7)

(4, 5)

(4, 7)

(5, 6)

(6, 7)

matrixes:

(1) object(s)

=0= (4)

(1.0, 0.0, 0.0, 0.0)

(0.0, 1.0, 0.0, 0.0)

(0.0, 0.0, 1.0, 0.0)

(0.0, 0.0, 0.0, 1.0)

I highlighted the LIST of numbers above so that we can see it more clearly.

NOTE: Here we are reading the CUBE as Vertices, Edges, Matrixes. Remember that you also pass on the Polygons, or the DATA that makes Faces/Polygons of the mesh.

BREAKING DOWN CUBE DATA

If I were to elaborate the date using my own words, it will be like this: Cube ingredient can be broken into these DATA:

Vertices (LIST of 8 values)
Edges (LIST of 12 values)
Matrixes (LIST of 4 values)

Each Vertice data have 3 FLOAT values, each value representing position in XYZ, in relation to the Matrix.

Each Edge data have 2 INT values, contain pair information on which Vertice Index connected to which.

Matrix data is storing information about Translation, Rotation, Scale, and Rotation Axis.

In Sverchok, just like in Rhino-Grasshopper, we are able to generate or supply our own data to pass on and recreate those DATA STRUCTURE required to make an object parametrically or procedurally.

This means, you can very easily and visually generate LIST of 3 values to represent POINT POSITIONS in XYZ. Or all kind of DATA in STRUCTURE that you like it to be for your own purpose.

Cube Data above is a simple example.

What interesting is when we start to MERGE, MIX, WRAP and JOIN data, you will encounter lots of interesting thing. Sometimes you need to FLATTEN the DATA Tree. Separate 3 values data into each separate value and then combine it back.

EXAMPLE: CREATING RANDOM VALUES XYZ

Generating a single TUPLE with 3 values (Vector) using Sverchok.

vertices:

(1) object(s)

=0= (1)

(0.7954123484458212, 0.9915130893390923, 0.3247524107944588)

unknown data:

None

matrixes:

None

Generating a LIST of some TUPLES with 3 values:

vertices:

(1) object(s)

=0= (10)

(0.8800991152954115, 0.2869275644617431, 0.4621446386044473)

(0.1707271599919563, 0.2141409389917276, 0.4621446386044473)

(0.6958015463360391, 0.6714156319278667, 0.4621446386044473)

(0.1252005547937043, 0.7170644202392952, 0.4621446386044473)

(0.1131273587904863, 0.2736442119694806, 0.4621446386044473)

(0.6394349933329605, 0.3203943496951187, 0.4621446386044473)

(0.3210433830317321, 0.4042363684941612, 0.4621446386044473)

(0.2363263605284892, 0.456282186215196, 0.4621446386044473)

(0.7362634528968245, 0.3493000706331532, 0.4621446386044473)

(0.6090024422764354, 0.3397017948731432, 0.4621446386044473)

unknown data:

None

matrixes:

None

BRINGING THOSE DATA INTO PYTHON

DATA is just DATA.

We can generate those data above in Python obviously. But what exciting here is the fact that we are creating and generating those DATA visually using Sverchok node-tree inside Blender. It is all visual.

Within Sverchok, we can modify, sort, join, and do all sort of interesting data manipulations. Mostly, we will care on how the data will transform back into Mesh geometry that we can use in normal Blender environment.

Current Sverchok is great for modeling. It is not yet directly connected to Blender typical environment, in which we are probably tempted to use Sverchok for Procedural Animation, Procedural Shading, etc.

Don't limit yourself. Either way, those DATA, or NUMBERS, can be exported or imported as we please.

Knowing this truth, we can basically bend the spoon like NEO in the MATRIX.

When we use Viewer Text Node in Sverchok, we already spit out a readable DATA for Python. Those DATA is separated by LINE, which is not as simple as copy-pasting to create actual list.

Let say, I want to do this:

"Generate random 3 values list using Sverchok, and use that data to create bunch of cubes randomly positioned colored with those values."

Random XYZ => Cube Positions

Random XYZ => Cube Colors

That can be done in Python. Is it easy? Well... it depends, but every step as logical as preparing ingredient for cooking something.

It actually took me a bit of time and I found problems here and there.

STEP 001

Prepare and generate the data in Sverchok. We have done this already simply copy paste list of random 3 values and save it as normal text file (TXT). For example, my data looks like below:

(0.6646910631367325, 0.908184595140585, 0.559640197467031)

(0.6259886197350288, 0.0009782863450561, 0.0635360130887022)

(0.1702762851837583, 0.0303247289132893, 0.4000292985038622)

(0.4247105715299129, 0.0239215297953929, 0.7871877036098419)

(0.253353935095027, 0.4481245252973097, 0.2827014293529476)

(0.329842697601205, 0.8651187090911354, 0.843854411030971)

(0.2707690256035583, 0.7882956210312217, 0.2807732651216759)

(0.4629872528241108, 0.6491480656266136, 0.5343794408560223)

(0.927270058247242, 0.7755588725338894, 0.4961782330878063)

(0.0655122421709582, 0.623794902689959, 0.5508732151641716)

STEP 002 - Import TXT DATA

We have out TXT data to be imported back into Blender to be used to create Cube Positions and Color. How do we do that using Python? Below is one way to do it.

# Import file

f = open('\\Users\Jimmy\Desktop\listData.txt')

lines = f.readlines()

f.close()

# Read each line one by one

# while converting each into normal tuple

# and appending into list I want

myList = []

counter = 1

for line in lines:

myList.append(eval(line))

#print(counter, line)

counter += 1

print(myList)

I did not come up with that in one go, I actually search around for solution using Google:

How to open TXT file in Python? open() and close()
How to read file line by line in Python? readlines()
When each line is read, I am getting String of values, how to convert those String DATA into normal TUPLE? Apparently eval() can be used as long the String is known to be a valid Tuple structure
Our Tuple is (x, y, z) so all good
We finally get our LIST containing 3 values
We can pass it on

STEP 003 - Use LIST DATA

From here, it is totally up to you what you want with the DATA.

Bend the spoon if you like.

However, hold your Force, here we just want to create bunch of Cubes, each Cube has POSITION and COLOR based on the DATA values we originally generated in Sverchok, which is now just a LIST DATA in Python.

Python can do the rest with Blender Python (BPY).

# From that list

# Create bunch of cubes stacked in Z position

# and Color them according to List tuple of Random Values

import bpy

for i, col in enumerate(myList):

bpy.ops.mesh.primitive_cube_add(view_align=False, enter_editmode=False, location=(10 * col[0], 10 * col[1], 10 * col[2]), rotation=(0, 0, 0))

bpy.context.object.color = (col[0], col[1], col[2], 1)

print (i, col)

Here, today, you have learn how we can see DATA and start thinking about DATA STRUCTURE to then used to MANIPULATE the DATA.

Now, you see how much "easier" it is to generate, modify, passing DATA in Sverchok?

Of course in Python it is "easy" as well, if you used to working with DATA in Python, just a lot more typing and Wing-Chun bashing that keyboard. In Sverchok we connect the node. If there is an error, we get nothing. In Python, we probably get error message.

It is not actually that much different, in Sverchok we use Viewer Text and SEE the DATA on Text Editor. In Python, we usually PRINT out the DATA and read what is happening.

Sverchok indeed comes with some useful Sverchok nodes to manipulate data which is useful to do parametric/procedural modeling.

Nikita aka NIKITRON on Blender Artist tried to explain DATA STRUCTURE to me:

- use data level slider to get level right

- if you need increase level = use join node with "wrap" activated.

- If you need reduce data level = use "List Levels" node to reduce

Anyways, this is when I asked question "Why LIST LENGTH" did not give me DATA that can be passed on to Random node? The answer is below and it is something to do with data structure.

TO GET LENGTH (TOTAL NUMBER OF EDGES):

Do not try to bend the spoon — that's impossible. Instead, only try to realize the truth: there is no spoon. - THE MATRIX

LINKS

http://www.rhino3d.com/download/Rhino/4.0/EssentialMathematicsSecondEdition

PYTHON / Additional Examples and Usages of XML

In previous post, I talked a lot about XML and probably some of you got confused on how we can really use this XML writing and reading knowledge to do export import of data.

This is fine because with the XML Read and Write, you really have a lot of concepts to digest:

Using Element Tree XML built in module
Concept of XML: tag, element
Using File function of Python to write out to file
Using and constructing data that is like Dictionary: keys, items, values
Data type conversions
Understanding of Path Location of XML file in your Operating Systems

This XML concept is also still fairly new to me and I only understand the basic to write out the XML style format and read it back in.

Take it easy and slowly with this XML. See the pattern.

However as we know, even the basic knowledge can be really useful. I will further expand the idea from previous post. Feel free to ask me questions if you don't have clue what I am talking about.

DIY Write & Read of XML

For this, I will start by copying and pasting the template construct from previous post on XML and modify it slightly each time for each example below. It would be better to understand the xml.etree module, but we could also simply use the construct as long you know what it does in the background.

We will definitely use some simple BPY to collect information and data we needed to export about the 3D scenes. Our focus is on how the data can flow out of Blender and bring it in back into Blender or to other 3D packages.

Remember that Python scripting, like any other programming language, is something you learn and master by actually practicing it and making mistake and fixing it.

If you understand my style of blog writing, I actually improvise while writing. Often I do not actually plan the topic. This blog is more like Journal. I explore certain feature, I am testing and documenting each steps and then I will write it down. I guess you could do the same thing.

Hopefully with this examples, you can start to get curious and try your own XML export import.

It will get easier every time you try it yourself. My examples are there to assist you. Try doing the same thing without looking at my examples and you will start to understand.

Keep digging the Python language and Python in Blender, it will be worth it.

EXAMPLE 1: XML Transform Information (Position, Rotation, Scale) of objects

The idea is to get the Name, Translations, Rotations and Scales data of every selected objects and writing it out as XML. Below is such example of code. It can be optimized, but below should work and the code is pretty easy to read.

Maybe I have bunch of Boxes in random Position, Rotation, Scale in Blender that I want to export out as XML data.

WRITE OUT:


'''
Taken from: Blender Sushi Blog
Code by:  Jimmy Gunawan
Last update: 20130420
'''





import bpy
from xml.etree import cElementTree as ElementTree

# PRETTY PRINT
# function to pretty print the XML code
def prettyPrint(element, level=0):
    '''
    Printing in elementTree requires a little massaging
    Function taken from elementTree site:
    http://effbot.org/zone/element-lib.htm#prettyprint

    '''
    indent = '\n' + level * '  '
    if len(element):
        if not element.text or not element.text.strip():
            element.text = indent + '  '

        if not element.tail or not element.tail.strip():
            element.tail = indent

        for element in element:
            prettyPrint(element, level + 1)

        if not element.tail or not element.tail.strip():
            element.tail = indent

    else:
        if level and (not element.tail or not element.tail.strip()):
            element.tail = indent

    return element

### Collecting some information data about the scene ###

# In this example we collect the selected object Transforms
# aka Position (Translation/Location), Rotation, and Scale XYZ
# and writing out the data as XML format

# Create ROOT node
rootNode = ElementTree.Element('world')

# Get Selected Objects
selected_objects = bpy.context.selected_objects

for object in selected_objects:
    myLocs = object.location
    myRots = object.rotation_euler
    myScales = object.scale
    
    # Create node = CHILD element
    elementNode = ElementTree.Element('{object}'.format(object=object.name))
    
    # Parent CHILD node to ROOT
    rootNode.append(elementNode)

    # Adding attribute translations
    elementNode.attrib['TX'] = str(myLocs[0])
    elementNode.attrib['TY'] = str(myLocs[1])
    elementNode.attrib['TZ'] = str(myLocs[2])
    
    # Adding attribute rotations
    elementNode.attrib['RX'] = str(myRots[0])
    elementNode.attrib['RY'] = str(myRots[1])
    elementNode.attrib['RZ'] = str(myRots[2])
    
    # Adding attribute scales
    elementNode.attrib['SX'] = str(myScales[0])
    elementNode.attrib['SY'] = str(myScales[1])
    elementNode.attrib['SZ'] = str(myScales[2])

    # Adding text
    elementNode.text = '{object}'.format(object=object.name)


### WRITING OUT IN A CLEAN WAY IS A BIT COMPLEX... WE USE THE PRETTY PRINT

# Pretty print the rootNode
prettyPrint(element=rootNode)

# We need to convert to STRING before we can see the whole thing
xmlText = ElementTree.tostring(rootNode)

# FINAL RESULT, we need to append the HEADER
print ( '<?xml version="1.0" ?>\n{xmlText}'.format(xmlText=xmlText) )

### NOW WE LIKE TO WRITE OUT THE XML TO FILE
outputPath = r'C:\pythonInBlender\transOut.xml'

fileObject = open(outputPath, 'w')
fileObject.write('<?xml version="1.0" ?>\n' + xmlText.decode('utf8'))
fileObject.close()

print('XML file has been written in here: {outputPath}'.format(outputPath=outputPath))

THE XML:

READ IN:


'''
Taken from: Blender Sushi Blog
Code by:  Jimmy Gunawan
Last update: 20130420
'''





import bpy

from xml.etree import cElementTree as ElementTree

xmlPath = 'C:/pythonInBlender/transOut.xml'

xmlRoot = ElementTree.parse(xmlPath).getroot()


def applyTransform():

    for element in xmlRoot:

        # Get the NAME of objects
        objectName = element.tag
        print(objectName)
        
        # Tell Blender to get the current object data by its name
        object = bpy.data.objects['{objectName}'.format(objectName=objectName)]
        print(object)

        # Get all element keys
        myKeys = element.keys()
        # print(myKeys)

        # Get Position, Rotation and Scale data            
        posX, posY, posZ = float(element.get('TX')), float(element.get('TY')), float(element.get('TZ'))
        rotX, rotY, rotZ = float(element.get('RX')), float(element.get('RY')), float(element.get('RZ'))
        scaleX, scaleY, scaleZ = float(element.get('SX')), float(element.get('SY')), float(element.get('SZ'))

        # Apply Position, Rotation and Scale data to corresponding object by its name
        object.location = posX, posY, posZ
        object.rotation_euler = rotX, rotY, rotZ
        object.scale = scaleX, scaleY, scaleZ
            
applyTransform()

The code above is very specific about Object Name and that is just one way we can do this. If for the same objects we reset their position, we can then run the script and magically put them back into positions specified from XML.

The above example is not particularly amazing example because we can achieve the same thing using DAE Collada Export Import, for example. Although with DAE, we usually have limited options and flexibility.

We could bring the XML data to other 3D package and actually do ANYTHING with it. Keep in mind that the World XYZ Orientation might be different in other package.

This kind of code is probably good for saving out "Presets"or maybe "Character Poses". I have not looked at it deeply. It has potential for Assembly script. Although Blender File System and Referencing are actually already pretty cool and we can probably do the script based on that system. But if we want to bring it outside and inside other package, then XML seems to be nice.

From the above basic example, we can take the concept to place some Monkeys in the 3D scene, if we can think of every Cubes as Bounding Box:

READ IN, BUT CAST AS MONKEYS:


'''
Taken from: Blender Sushi Blog
Code by:  Jimmy Gunawan
Last update: 20130420
'''





import bpy

from xml.etree import cElementTree as ElementTree

xmlPath = 'C:/pythonInBlender/transOut.xml'

xmlRoot = ElementTree.parse(xmlPath).getroot()


def createMonkeyFromData():

    for element in xmlRoot:

        objectName = element.tag

        if 'TX' in element.keys():

            # Get Position Data            
            posX, posY, posZ = float(element.get('TX')), float(element.get('TY')), float(element.get('TZ'))

            # Create Suzanne based at position XYZ
            bpy.ops.mesh.primitive_monkey_add( location=(posX, posY, posZ) )
            
            monkey = bpy.context.selected_objects[0]
            print (monkey)
            
            # Get Rotation and Scale data
            rotX, rotY, rotZ = float(element.get('RX')), float(element.get('RY')), float(element.get('RZ'))
            scaleX, scaleY, scaleZ = float(element.get('SX')), float(element.get('SY')), float(element.get('SZ'))
            
            # Rotate and Scale monkey accordingly from data
            monkey.rotation_euler = rotX, rotY, rotZ
            monkey.scale = scaleX, scaleY, scaleZ
            
 
createMonkeyFromData()

NOT EXACT DIMENSION SIZE
We apply the Transform data from the Cubes into the Monkeys, although is not really the accurate Bounding Box or Dimension data. Because some parts of the monkey is poking out.

If you are questioning about it.We could export out the Dimension data as XML from the first place and apply it to the monkeys to get more accurate result.

Example on how to get dimension data of object to pass on:
bpy.data.objects[{object_name}].dimensions
bpy.data.objects['Cube'].dimensions

EXAMPLE 2: Reading In Custom Attribute Data

Let say we have XML data with arbitrary Custom Attribute and values. We did not create the date from Blender. Someone else did it. Maybe we grab the data from the Internet.

The XML data can be anything:
1. Yahoo! Weather XML data
City Location
Temperature High
Temperature Low
Today's Date
2. Facebook XML data
Person's Name
Person's Photo
Person's Age
Person's Likes
3. Google Map data
Latitude
Longitude
Name of State
etc
What we can do is simply READING those data and then visualizing it in Blender. 

A simple example, XML like below:

<world>

<Object1 COLOR="red" SIZE="5.0"/>

<Object2 COLOR="green" SIZE="2.0"/>

<Object3 COLOR="blue" SIZE="0.5"/>
</world>

OR MORE PROPERLY:

<?xml version="1.0" ?>
<world>
  <Object1 COLOR="red" SIZE="5.0"></Object1>
  <Object2 COLOR="green" SIZE="2.0"></Object2>
  <Object3 COLOR="blue" SIZE="0.5"></Object3>
</world>

(Save the bottom one as customData.xml.)

Ok, let say from those data, I will apply it to the Monkey.001, Monkey.002, and Monkey.003 objects in the 3D scene that I have prepared.

The Suzanne Three Sisters
What I am trying to do is to tell Blender:
Object1 --> Monkey.001 --> apply SIZE, apply MATERIAL with Diffuse Color = COLOR
Object2 --> Monkey.002 --> apply SIZE, apply MATERIAL with Diffuse Color = COLOR
Object3 --> Monkey.003 --> apply SIZE, apply MATERIAL with Diffuse Color = COLOR

The script can be as below:


READ IN CUSTOM XML
'''


Taken from: Blender Sushi Blog


Code by:  Jimmy Gunawan


Last update: 20130420


'''





import bpy

from xml.etree import cElementTree as ElementTree

xmlPath = 'C:/pythonInBlender/customData.xml'

xmlRoot = ElementTree.parse(xmlPath).getroot()


def applyColorToMonkey():

    for element in xmlRoot:
        
        # Create Materials with Color based on the NAME  
        colorname = element.get('COLOR')
        mat = bpy.data.materials.new(name='{colorname}'.format(colorname=colorname))
        
        # Create our own custom dictionary of color
        colorDict = {'red':[1.0,0.0,0.0], 'green':[0.0,1.0,0.0], 'blue':[0.0,0.0,1.0]}
        
        # Assign specified Diffuse Color into Material
        myColor = colorDict.get(colorname)
        bpy.data.materials[colorname].diffuse_color = myColor
        
        # Get the NAME of objects
        objectName = element.tag
        objectNumber = int(objectName.strip('Object'))
        monkeyName = 'Monkey.{objectNumber:03}'.format(objectNumber=objectNumber)
        
        print (monkeyName)
        
        # Tell Blender to get the monkey data by its name
        object = bpy.data.objects['{monkeyName}'.format(monkeyName=monkeyName)]
        
        # Assign our Material to corresponding Monkey
        object.data.materials.append(mat)
            
def applyScaleToMonkey():

    for element in xmlRoot:

        # Get the NAME of objects
        objectName = element.tag
        objectNumber = int(objectName.strip('Object'))
        monkeyName = 'Monkey.{objectNumber:03}'.format(objectNumber=objectNumber)
        
        print (monkeyName)
        
        # Tell Blender to get the monkey data by its name
        object = bpy.data.objects['{monkeyName}'.format(monkeyName=monkeyName)]
        
        print(object)

        # Get SIZE data and assign it to Scale Variables        
        scaleX, scaleY, scaleZ = float(element.get('SIZE')), float(element.get('SIZE')), float(element.get('SIZE'))

        # Apple Scale to Monkeys
        object.scale = scaleX, scaleY, scaleZ


applyColorToMonkey()
applyScaleToMonkey()







If the above code run as planned, we are getting something like below. This is the power of XML and Python and Python in Blender. 

We should be able to:
Write XML data (free the data to the ouside)
Read XML data (from any source)
Process the data
Apply the data

The code above can be further cleaned up and probably expanded
Error Checking
Check if object exist
Making sure Blender does not keep generating same Material Name when we run the script multiple times.
FURTHER ON WITH XMLSee if you can play with the idea further more:
Assigning "specific Texture Name" for each Material to be assigned into "specific" objects
Procedurally generate XML data 
Can we perhaps use other tool like Processing to generate XML and then bring it into Blender?
Turn it into Animation, let say if we have XML data in corresponding to Frame Number.
Find some XML data from Internet and bring it into Blender and visualize it.
Almost "Real time" XML data with refresh?

OTHER VIDEO TUTORIAL RELATED TO XMLLuiz Kruel @Eat3D
http://eat3d.com/free/maya_python

Based from his video tutorial, below is the converted Blender Script that does the XML export using MINIDOM.

from xml.dom.minidom import Document
import bpy

doc = Document()

root_node = doc.createElement("Blender Scene Info")
doc.appendChild(root_node)

# Get selected objects
selection = bpy.context.selected_objects

# Just print out selected objects name
for num, item in enumerate(selection):
    print (num, item.name)

# Add into XML tree:

for object in selection:
    object_node = doc.createElement(str(object.name))
    root_node.appendChild(object_node)

    object_locX = object.location.x
    object_locY = object.location.y
    object_locZ = object.location.z
    object_node.setAttribute("locationX", str(object_locX))
    object_node.setAttribute("locationY", str(object_locY))
    object_node.setAttribute("locationZ", str(object_locZ))
    
print()
print()
print()

xml_file = open("C:/test.xml", "w")
xml_file.write(doc.toprettyxml())
xml_file.close()

print ( doc.toprettyxml() )

PYTHON / Advance Import and Export Using XML

Bunch of Cubes from Maya, imported into Blender (Color, Position) via XML and dynamically animated using Blender Bullet and rendered out.

PREREQUISITE

Python basics
Python data type, especially Dictionary
cElementTree module
File Read and Write

We will be looking at a bit of theory of XML and its practical usage.

PROGRAMMING AND DATA

The more one learns about programming in Python (in Blender or other 3D packages), the more one can see the pattern that the whole thing is really about "data management" and "data processing" and "data visualization":

How to retrieve data
How to process data
How to store data
How to export and import data
How to display or present the data

As "boring" as those may sound for artists at first, data is apparently can further be processed and presented elegantly in a creative ways. Lots of raw data just means more opportunity for us (artists) to visualize it in a beautiful kind of way.

3D artists (like you and me) are lucky from the very beginning to work visually with computer data from the very beginning (at least in the last 20+ years). All we have to deal is the Interface. Still there is a sweet and sad labor of hours in doing what you are doing :)

However, if we really want to dig deeper with Computer Graphics, it is nice to be able to read raw data and process them in a magical kind of way.

In this post, I will be attempting to do an introduction to XML, XML Writing and Reading. I am not from programming background and writing this post is quite strange to myself as well. Anyhow, I feel like writing and sharing with you all.

IT WAS CSV

I have written a bit about CSV (Comma Separated Value) data and Blender export-import using CSV a while ago. At the time I was still clueless about Python and programming. I felt that CSV is really simple and easy to understand so that was my reason to use CSV.

Sooner or later we will realize that CSV is probably limited in term of what we can "store" and "parse" from it. It does not have much of STRUCTURE apart from a long list of strings and numbers and line numbers. It can represent ROWS and COLUMN type of data, but not so much of HIERARCHY data.

See how CSV is parsed inside program like NodeBox or Houdini. It's still pretty cool:
CSV -> TABLE -> VISUALIZATION
http://blendersushi.blogspot.com.au/2012/12/python-importing-and-visualizing-data.html

\BLENDER CSV F-Curve Import
http://blenderartists.org/forum/showthread.php?209181-A-Script-to-Import-a-CSV-File-and-Create-F-Curves-(for-Blender-2-5x-or-later)

So then, what is the alternative of CSV? Just a few days ago, I learned something new (yes, new for programming newbie like me!) called XML.

I think this XML is really worth looking.

Let see what can we do with XML using Python inside Blender.

What is XML?

XML stands for eXtensible Markup Language. Apart from this long name, I think the best description and what XML really is, can be read here:
http://www.w3schools.com/xml/xml_whatis.asp
http://www.w3schools.com/xml/xml_tree.asp#gsc.tab=0

The simple way I see XML is a kind of data format with tree-like hierarchy structure. XML is human readable, just like CSV, but when we look at the XML data, it is more like a Tree with Branches and Leaves.

<root>
<object1 attribute1 = "value" attribute2 = "value" attribute3 = "value">
<object2 attribute1 = "value" attribute2 = "value" attribute3 = "value">
<object3 attribute1 = "value" attribute2 = "value" attribute3 = "value">
</root>

WHAT XML CONSIST OF?

root node, child node, grand child node, and so on. it always has root.
tag
element(s)
attribute(s)
attribute value(s)
text

If I continue on digging on XML data structure without some examples probably we all can go to sleep. But I will try show you some interesting things soon.

Since XML itself is a little boring to look at and actually does nothing, why do we even need to look at this? One would rather spend time doodling inside Blender already!

Be patient, my friend.

Apparently the data stored in XML structure is easily accessible via Python, just like collection of Dictionary and List. It is a lot like Library actually.

This means with XML, we can easily WRITE OUT and READ IN data. And this is a cool way we can TRANSFER DATA between any software packages or tools that supports Python. We can make them talk to one another as we can see soon.

The data inside could be anything and we can organize the data the way we like it, because with XML we can create our own custom "tags" and store the data the way we like within the XML structure.

PYTHON AND XML

Python provides many MODULE to work efficiently with XML data.
http://docs.python.org/2/library/xml.etree.elementtree.html

We will be using cElementTree module instead of DOM module.
http://effbot.org/zone/element.htm
http://www.bigfatalien.com/?p=223
http://eli.thegreenplace.net/2012/03/15/processing-xml-in-python-with-elementtree/

SIMPLE VS COMPLEX XML STRUCTURE

There are simple and there are complex XML structure.

A simple XML structure is when we only have 1 level of child under the root/parent.

<?xml version="1.0" ?>
<character>
<pCube1 rotateX="0.0" rotateY="0.0" rotateZ="0.0" scaleX="1.0" scaleY="1.0" scaleZ="1.0" translateX="0.0" translateY="6.36026310449" translateZ="0.0" visibility="True" />
<pCube2 rotateX="0.0" rotateY="0.0" rotateZ="0.0" scaleX="1.0" scaleY="1.0" scaleZ="1.0" translateX="0.0" translateY="12.8163721197" translateZ="0.0" visibility="True" />
<pCube3 rotateX="0.0" rotateY="0.0" rotateZ="0.0" scaleX="1.0" scaleY="1.0" scaleZ="1.0" translateX="0.0" translateY="12.8163721197" translateZ="0.0" visibility="True" />
</character>

OR

<?xml version="1.0" ?>
<character>
<MASTER rotateX="0.0" rotateY="0.0" rotateZ="0.0" scaleX="1.0" scaleY="1.0" scaleZ="1.0" translateX="0.0" translateY="0.0" translateZ="0.0" />
<simplebot_lf_knee_pv_ctrl lfLegIkCtrl="0.0" snapKnee="0.0" translateX="0.0" translateY="0.0" translateZ="0.0" />
<simplebot_rt_knee_pv_ctrl rtLegIkCtrl="0.0" snapKnee="0.0" translateX="0.0" translateY="0.0" translateZ="0.0" />
<simplebot_rt_foot_ctrl ankleStretch="0.0" fkVis="False" ikFkBlend="1.0" ikVis="False" kneeStretch="0.0" legStretch="0.0" />
<simplebot_lf_foot_ctrl ankleStretch="0.0" fkVis="False" ikFkBlend="1.0" ikVis="False" kneeStretch="0.0" legStretch="0.0" />
<COG rotateX="0.0" rotateY="0.0" rotateZ="0.0" translateX="0.0" translateY="0.0" translateZ="0.0" />
<L_FOOT rotateX="0.0" rotateY="0.0" rotateZ="0.0" translateX="0.0" translateY="0.0" translateZ="0.0" />
<simplebot_lf_heel_ik_ctrl ballTwist="0.0" footBreak="40.0" footRoll="0.0" legTwist="0.0" maxStretch="3.0" pvControl="True" stretchyLeg="0.0" toeRoll="0.0" toeTwist="0.0" />
<R_FOOT rotateX="0.0" rotateY="0.0" rotateZ="0.0" translateX="0.0" translateY="0.0" translateZ="0.0" />
<simplebot_rt_heel_ik_ctrl ballTwist="0.0" footBreak="40.0" footRoll="0.0" legTwist="0.0" maxStretch="3.0" pvControl="True" stretchyLeg="0.0" toeRoll="0.0" toeTwist="0.0" />
<REye_ctrl translateX="0.0" translateY="0.0" translateZ="0.0" />
<LEye_ctrl translateX="0.0" translateY="0.0" translateZ="0.0" />
<JG_Eye_GP eyeZ="0.0" translateX="0.0" translateY="0.0" translateZ="0.0" twist="0.0" />
</character>

They may be slightly hard to read, but still human readable and our brain can see the pattern of what it is trying to do.

A complex XML structure is when we store every values inside child of child. Here is example I took from:
http://www.w3schools.com/dom/dom_nodes.asp

<bookstore>
<book category="COOKING">
<title lang="en">Everyday Italian</title>
<author>Giada De Laurentiis</author>
<year>2005</year>
<price>30.00</price>
</book>
<book category="CHILDREN">
<title lang="en">Harry Potter</title>
<author>J K. Rowling</author>
<year>2005</year>
<price>29.99</price>
</book>
<book category="WEB">
<title lang="en">Learning XML</title>
<author>Erik T. Ray</author>
<year>2003</year>
<price>39.95</price>
</book>
</bookstore>

I reckon that the complex XML is more "correct", but then the simple XML, the single liner that can be accessed as Dictionary is sufficient for us most of the time. It is really up to us how we like to store and organize the XML.

PS: Shaun Friedberg suggested the simple XML structure. Shaun is actually the one introducing me to this XML stuff (in Maya).

CREDIT NOTE:
Originally some of the contents I talked about in here is from the Python course taught by Shaun Friedberg
http://www.linkedin.com/in/pyrokinesis

WRITING XML (EXPORT XML DATA)

Whatever value we can print() in Python, we can easily pass that as XML lines.

We could type the XML lines by hand manually, but that is silly and we definitely will not do that unless we are a robot. We could of course edit the XML by hand too if we like, but that is also something we don't usually do.

We certainly use Python and LOOP to write XML.

Some module to help writing and parsing XML:

ElementTree
cElementTree
dom

It will be beneficial if you search and study some Python example to write and read XML using the above modules. In our case, we will be using cElementTree, because it seems to be the fastest.

BLENDER SUSHI SCRIPT: Writing Out XML Doodle Script


'''


Preparing XML construct and Writing out XML


Based on teaching by Shaun Friedberg


Edited by Jimmy Gunawan (Blender Sushi)


'''





from xml.etree import cElementTree as ElementTree

# PRETTY PRINT
# function to pretty print the XML code
def prettyPrint(element, level=0):
    '''
    Printing in elementTree requires a little massaging
    Function taken from elementTree site:
    http://effbot.org/zone/element-lib.htm#prettyprint

    '''
    indent = '\n' + level * '  '
    if len(element):
        if not element.text or not element.text.strip():
            element.text = indent + '  '

        if not element.tail or not element.tail.strip():
            element.tail = indent

        for element in element:
            prettyPrint(element, level + 1)

        if not element.tail or not element.tail.strip():
            element.tail = indent

    else:
        if level and (not element.tail or not element.tail.strip()):
            element.tail = indent

    return element



# Create ROOT node
rootNode = ElementTree.Element('root')

### CREATE FIRST LINE OF ELEMENT

# Create node = CHILD element
elementNode = ElementTree.Element('node')

# Parent CHILD node to ROOT
rootNode.append(elementNode)

# Adding attribute
elementNode.attrib['exampleAttribute'] = 'example value'

# Adding more attributes
elementNode.attrib['color'] = 'Blue'
elementNode.attrib['smell'] = 'Ocean'
elementNode.attrib['eat'] = 'Sushi'

# Adding text
elementNode.text = 'this is an example text'

### CREATE SECOND LINE OF ELEMENT

# Create node = CHILD element
elementNode = ElementTree.Element('node2')

# Parent CHILD node to ROOT
rootNode.append(elementNode)

# Adding attribute
elementNode.attrib['exampleAttribute'] = 'example value'

# Adding more attributes
elementNode.attrib['color'] = 'Red'
elementNode.attrib['smell'] = 'Fire'
elementNode.attrib['eat'] = 'Spice'

# Adding text
elementNode.text = 'this is second example text'



### WRITING OUT IN A CLEAN WAY IS A BIT COMPLEX... WE USE THE PRETTY PRINT

# Pretty print the rootNode
prettyPrint(element=rootNode)

# We need to convert to STRING before we can see the whole thing
xmlText = ElementTree.tostring(rootNode)

# FINAL RESULT, we need to append the HEADER
print ( '<?xml version="1.0" ?>\n{xmlText}'.format(xmlText=xmlText) )

### NOW WE LIKE TO WRITE OUT THE XML TO FILE
outputPath = r'C:\pythonInBlender\test.xml'

fileObject = open(outputPath, 'w')
fileObject.write('<?xml version="1.0" ?>\n' + xmlText.decode('utf8'))
fileObject.close()

print('XML file has been written in here: {outputPath}'.format(outputPath=outputPath))

RESULT:

Additional Reference:

http://washort.twistedmatrix.com/2010/11/unicode-in-python-and-how-to-prevent-it.html

READING XML (IMPORT XML DATA)

Reading data is much simpler than writing. Well, at least it seems so. Writing data seems to be more complicated at first glance.

What we need to care is what we are going to do with the data after being exported. We also need to do some Data Conversion sometimes because all the data seems to come in as String, where sometimes we need data as Integer or Float for further processing.

The data comes in as Dictionary type of data. Hopefully you have some understanding of Dictionary in Python. I am also still new to all this and it is a matter of getting the KEY and VALUE from each ITEM per ELEMENT.

BLENDER SUSHI SCRIPT: Reading In XML Doodle Script



import bpy


from xml.etree import cElementTree as ElementTree





xmlPath = 'C:/pythonInBlender/mayaOut.xml'


xmlRoot = ElementTree.parse(xmlPath).getroot()





def createMonkeyFromData():


    for element in xmlRoot:





        objectName = element.tag


        if 'translateX' in element.keys():


            #print( 'Attribute called TranslateX: {name}'.format(name=element.get('translateX')) )


            locX = float(element.get('translateX'))


            locY = float(element.get('translateY'))


            locZ = float(element.get('translateZ'))


            bpy.ops.mesh.primitive_monkey_add( location=(locX, locY, locZ) )





createMonkeyFromData()

import bpy
from xml.etree import cElementTree as ElementTree

xmlPath = 'C:/pythonInBlender/mayaOut.xml'
xmlRoot = ElementTree.parse(xmlPath).getroot()

def createMonkeyFromData():
for element in xmlRoot:

objectName = element.tag
if 'translateX' in element.keys():
#print( 'Attribute called TranslateX: {name}'.format(name=element.get('translateX')) )
locX = float(element.get('translateX'))
locY = float(element.get('translateY'))
locZ = float(element.get('translateZ'))
bpy.ops.mesh.primitive_monkey_add( location=(locX, locY, locZ) )

createMonkeyFromData()

BLENDER XML BUILDER?
There is a script called xmlbuilder.py inside Blender, I am curious what this actually does. I will check this when I have time. Or you can check it. Maybe it is something to do with DAE (Collada) in Blender.

In the mean time, I will use the WRITING XML and READING XML construct like above.

CASE STUDY: Maya to Blender via XML

Comparison of Python in Maya and Blender
I have to admit that Maya provides a lot more (shortcut) commands that allow users to quickly get ATTRIBUTE or VALUE data from one object (or node in Maya).

A little bit information:
At the current state, Python in Maya is well-known as "Wrapper" of MEL (Maya Embedded Language) programming. MEL has been there in Maya for years, before later Python came in.

MEL scripts itself has hundreds of handy scripts to access all kind of data from nodes. But, even so, Python handles "Strings" and "List" like in a really awesome ways, and this makes writing script is surprisingly easy and data flow is becoming a breeze in Maya.

It will be a little weird for people learning Python in Maya because they have to always look at the MEL first. They need to kind of learn MEL too, at least to really understand the functions and reading the history.

There is PyMEL from Luma Pictures that allow for more Pythonic way to access data, kind of the more correct way Python should implemented in Maya. This is more like how Python in Blender works. It is all Object Oriented (OOP).

There are some Python API that allows to access some Maya's core that normally only accessible using C++ programming. But it is pretty complicated and low level and I am not familiar with those yet.

I am still new to Python in Blender and Blender BPY stuff and still getting familiar with the construct. I am definitely not 100% sure with a lot of things BPY. Meaning, I have a lot of questions regarding the API itself.

Like I said earlier, it seems to be much easier to access DATA inside Maya.

Examples of some handy "commands" in Maya:

List
Select
Get Attribute
Set Attribute
Connect Attribute
List Attribute
xform

Blender Python "does not" come with all those kind of commands. BPY feels more "raw" in my opinion.

Users may actually write their own COMMANDS that can mimic all those handy tools that are given from Maya commands.

In Blender, you are accessing the data and value more directly. In Maya, they give some commands to access the data and value in more handy kind of way.

Exporting data From Maya To Blender using XML
Let's take a look at example that I tested the other day.

I am inside Maya and I have 3000 cubes created using Python in Maya, which was based on 3000 Particle POSITION and COLOR. The Particle itself inherited its Colors from Procedural RAMP texture.

You can read a bit more at my Maya blog:

http://mayaspiral.blogspot.com.au/2013/04/python-getting-particle-rgbpp-and-apply.html

In Maya, it is super easy to query ANY Attributes and Values of Particles, including custom attributes per Particle that user created.

In Blender, we can also easily get Particle Attributes such Position, Size, etc. but still pretty limited at the current state. Blender unfortunately cannot access Particle Color by its index. Not at the moment, but I think pretty soon in the future. Fingers crossed.

EXAMPLE: Blender Particle, get position, create cube at position
For example, if I want to create some bunch of cubes at Particle Position, we could so something like this:

import bpy

obj = bpy.context.active_object
ps = obj.particle_systems.active
#ps.particles[0].location

def printPosition():

for i in range(len(ps.particles)):
location = ps.particles[i].location
print(i, location)

# Create Cube At Position
bpy.ops.mesh.primitive_cube_add(location=(location[0], location[1], location[2]))
selectedObject = bpy.context.selected_objects
selectedObject[0].name = 'pixelcube.%s' % i
selectedObject[0].scale = 0.45, 0.45, 0.45

Now, the reasons I am doing the XML data export from Maya to Blender are below:

Because I want to get more familiar with Python and XML data export import
Because Maya's Bullet is currently sucks. It is very limited and slow.
Because Blender's Bullet is awesome and fast.
Because I am thinking of Pipeline where Blender can be the Renderer, Compositor, etc. And I like the practical Blender Fire and Smoke and some more VFX things.

And so I wrote script that does the exporting of some specifc datas from some nodes (every objects in Maya is nodes).

Basically from the 3000 cubes, I exported the COLOR RGB information. That is what I need. I could export the POSITION data as well, but for that I actually just use DAE Collada. Same thing.

How example XML data from Maya looks like:

Simple Tree View in Chrome

The actual XML Data

Importing data Into Blender From Maya using XML
As the data is now "outside" of Maya as XML, we can do anything with it.

I bring the 3000 cubes from Maya as Collada DAE, Blender import them fine with the node names in tact and including Material that was set in Maya. Make sure the Unit Size is checked.

I wrote the script below to READ the XML data.

XML, Dictionary, Strings
XML, when imported will come as Dictionary type of data. And it is also STRING type of data. I imported the RGB like this "[0.124, 0.55, 0.123]" and they are strings. Where the data expected supposed to be 3 values of Float.

In my script below, I have noob attempt to convert 3 values string into 3 values float.

BLENDER SUSHI SCRIPT: Importing RGB via XML (but you need to setup the XML)


import bpy
# import xml.etree.ElementTree as ET
from xml.etree import cElementTree as ET

def applyColor(object='pCube999', rgb = [1.0,0.0,0.0]):
    # Get specific object
    object = bpy.data.objects[object]
    
    # Create new vertex color
    object.data.vertex_colors.new()
    
    # Get Total Length of Vertex Colors for each selected meshes
    totalVertCol = len(object.data.vertex_colors[0].data)
    
    # Iterate over every mesh vertex color and give it a single colour
    for i in range(totalVertCol):
        object.data.vertex_colors[0].data[i].color = rgb

filepath = 'C://xml//test.xml'
tree = ET.parse(filepath)
root = tree.getroot()

print(root.tag)

# root = ET.fromstring(country_data_as_string)

for child in root:
    # Get node name
    nodeName = child.tag
        
    # Get Dictionary (including key and value)
    dict = child.attrib
    
    # Extract value from Dictionary
    rgb = dict.get('rgbVal')
    rgbClean = rgb.strip( '[]' )
    rgbList = rgbClean.split(',')
    rgbColor = float(rgbList[0]), float(rgbList[1]), float(rgbList[2])
    print (rgbColor)
    
    applyColor(object='{nodeName}'.format(nodeName=nodeName), rgb = rgbColor)

Thanks again to Taisuke Tanimura and his magic of list comprehension, there is a better way of doing that 3 values String RGB into correct 3 values FLOAT:

rgbList = [float(token) for token in rgbVal[1:-1].split(',')]

Big thanks to:

Shaun Friedberg
http://workshops.cgsociety.org/instructor.php?userid=14448
http://pyrokinesis.cgsociety.org/about/
http://www.imdb.com/name/nm3571084/
Taisuke Tanimura
http://www.imdb.com/name/nm2552359/
http://somewhatslanted.com/yoga/

Labels

Movies Post

Sports

Music

Business

Pictures

728x90 AdSpace

Travel

Fashion

Design

Movies

Recent Videos

Labels

Category

Slider

Recent Post

Games

Popular Posts

Pages

Find Us On Facebook

Links

About Me

Technology

Breaking News

DATA

DATA STRUCTURE

BREAKING DOWN CUBE DATA

EXAMPLE: CREATING RANDOM VALUES XYZ

BRINGING THOSE DATA INTO PYTHON

LINKS

DIY Write & Read of XML

EXAMPLE 1: XML Transform Information (Position, Rotation, Scale) of objects

WRITE OUT:

READ IN, BUT CAST AS MONKEYS:

EXAMPLE 2: Reading In Custom Attribute Data

READ IN CUSTOM XML

FURTHER ON WITH XML

OTHER VIDEO TUTORIAL RELATED TO XML

PREREQUISITE

PROGRAMMING AND DATA

IT WAS CSV

What is XML?

PYTHON AND XML

SIMPLE VS COMPLEX XML STRUCTURE

WRITING XML (EXPORT XML DATA)

BLENDER SUSHI SCRIPT: Writing Out XML Doodle Script

READING XML (IMPORT XML DATA)

BLENDER SUSHI SCRIPT: Reading In XML Doodle Script

CASE STUDY: Maya to Blender via XML

BLENDER SUSHI SCRIPT: Importing RGB via XML (but you need to setup the XML)

Big thanks to:

COMING SOON (HOPEFULLY)

XML to save poses and presets of objectsXML for assembly and layout between 3D packagesXML for saving animation and simulation data OTHER LINKS

MKRdezign

Contact Form

XML to save poses and presets of objects
XML for assembly and layout between 3D packages
XML for saving animation and simulation data

OTHER LINKS