Halloween Costume ideas 2015

PYTHON / Importing and Visualizing Data using Python in Blender

It has been quite a while since I posted that old post on Blender Python Scripting topic:
http://blendersushi.blogspot.com.au/2012/03/programming-one-can-script.html

I am keeping my eyes on that post and will continue to update it overtime as I gain more understanding of concept of programming, and how one can use programming inside 3D software like Blender, and how Python is utilized. I probably will re-write some of my approach in explaining Blender Python to you. It is only recently that I actually started to understand Python and programming a little bit better (than last time).

I will babble a little bit, but if you think you are a pro and want to skip, just go to subheading "PROGRAM START HERE". Do not miss "IF BUCKY CAN, YOU CAN" though if you are beginner in Python.

##########################################################
NOTE 2012.12.07:
I am looking for simplest "Syntax Highlighter" for Code Snippet. At the moment the Python Code written here looks pretty ugly and messy because the lines and tabs are not presented nicely as it supposed to). Anyhow, if you first copy-paste the code inside the box into a good text editor (Sublime, NotePad++), it should work. In Python, space and Tab often matters when writing code.
##########################################################

FROM FIRST STEP TO NEXT STEP
Ok, so that post should be "the first step" if you also want to get into programming. This post I am writing will be still touching BASIC of Python but also jumping  into INTERMEDIATE level.

Here, I am not writing Python theory per se, but this post will be kind of like written documentation to share some notes that I collected while I am studying Python and trying to find SOLUTION for PROBLEM.

WAY OF TEACHING AND CODING
I want to emphasize something that I learned recently:
Modern education is no longer about TEACHER and STUDENT, but more likely it is a COLLABORATION between people to think CREATIVELY in finding SOLUTION for PROBLEM. This is seriously applicable when you are learning 3D using Blender on your own, with helps of Online Community.

Learn to Google for information and read and filter forum postings. Do ask forum if you are completely stuck and cannot find solution anywhere else. Even nicer is to have some kind of meetup to talk and get active about your passion (Blender in this case). You are your own teacher.

I will try to be like a MENTOR. I do not know everything, but I will try showing you the way. Maybe you already have solution yourself though.

IF BUCKY CAN, YOU CAN
First thing first, I must share this with you, I stumbled into this awesome YouTube Channel full of interesting materials, owned by a guy named Bucky. Bucky teaches all kind of stuffs in his own way. Although most of the stuffs there are "basic" but this guy really teaches the basic really well. I actually would like to adapt his way of teaching for my future video tutorials.

Bucky also teaches Python Basics, here is a link to Playlist for snippet of videos explaining the foundation of Python:
http://www.youtube.com/course?list=ECEA1FEF17E1E5C0DA&feature=plcp

It is not related directly to Blender, but it will give you excellent introduction on Python.

Bucky is a great teacher, I salute people like this.

WHAT IS THE TASK YOU WANT TO DO USING PYTHON?
This is an important question once you get a hang of Python and programming a little bit better. What interest you the most?
  • Tools and UI creation
  • Data I/O (Input Output)
  • 3D Data Manipulation
  • Rigging and Animation
  • Visual Effects
  • Game (physics, behaviour, interaction)
(Although I sound like I know things, but I am also still figuring a lot of things. So please keep that in mind and bear with me,)

Whenever thinking about Blender Python, beginner programmers tends to what to accomplish big tasks. Even me, was kind of naively thinking to achieve complex stuffs.

Have you watched the movie "Jiro Dream of Sushi (2011)"?
http://www.imdb.com/title/tt1772925/

Well, that movie is basically a documentary about Master of Sushi called Jiro in Japan. Any beginner apprentice would like to work with the Master takes years (to learn how to prepare Squid perfectly) before allowed to touch Fish and create Sushi.

It will be frustrating. You will make thousands of mistakes.

Anyways, if you are stuck, always go few back steps, maybe you need certain "basic" skills in order to successfully complete that task.

In this post, I will attempt to show you this simple task that I manage to accomplish using Blender and Python. You will learn a lot just by accomplishing small task.

000. PROGRAM START HERE
Enough babbling around let's get started.

I bet Python can do a lot of stuffs, but for beginner like me, I have big interest in how Python can be used to bring DATA in and out from one place to another. In our context, let see how Blender can take data from outside and then process into something that you can render.

I seriously do not know how to do this, this morning. But now 7+ hours later, I finally understand and decided to spend some time writing this notes to share with you.

001. THE TASK IS...
To import data and create 3D visualization in Blender using Python.

002. WHAT IS THE DATA? 
Or in programming context, what type of data are we dealing here?

I cannot write all the theory regarding DATA TYPE, but however you can Google and read some articles or video tutorials on data type in programming in general.

Watch and look up for these terminologies:
  • String -> string of text
  • Float -> floating value of number eg. 14.233, 1.245
  • Integer -> round number eg. 5, 9, 1456
  • Boolean -> True or False
  • List --> you will deal with list a lot in Python, it is the strength of Python
  • Array
  • Tuple
  • Dictionary
003. PREPARE THE RAW DATA
For this example, I just use a simple Tabular Data in a Spreadsheet that I created inside Google Doc Spreadsheet. It looks like this:

Very simple row x column tabular data. The COLUMN x ROW data basically contains:
  • Name (text data)
  • Number (number data)
  • Color (text data)
Total 11 row of data.

How do we bring that data into Blender using Python?

There are probably many ways to do that, but one thing that I certain is using CSV (Comma Separated Value) file format.

From Google Docs, you simply to to File - Download As - CSV. You will then get a file in CSV format. It is basically a text format, but the data comes as rows of text, with comma separated value, like below:

helloworld,14,red
one,23,green
two,67,blue
three,43,blue
four,77,red
five,99,green
six,45,red
seven,32,blue
eight,78,green
nine,22,red
ten,73,blue

CSV file format is something that Python can process quite nicely using Python CSV Module. Anyhow, let's slowly see how this can be accomplish. Let's bring in this data into Blender using Python.

004. READING CSV
I learn how to read CSV by reading this blog post:

I like that blog a lot, and I think you will also find it useful. Lots of great snippet (short Python code) that you can study and use.

Basically, you need to import the CSV file.
file = csv.reader(open('\\Users\Jimmy\Desktop\data.csv', newline=''), delimiter=',')

Make sure that you write the correct directory path. I use Windows 7, so the data path is as below:
'\\Users\Jimmy\Desktop\data.csv'

Then you parse the data using CSV module.

The STARTER PYTHON SCRIPT inside Blender then goes like this:
import bpy # THIS IS A MUST TO IMPORT ALL BLENDER PYTHON MODULE
import csv # IMPORT CSV MODULE

# READ INPUT FILE

file = csv.reader(open('\\Users\Jimmy\Desktop\data.csv', newline=''), delimiter=',')

for idx, row in enumerate(file):
if idx<11: # this is hard coded at the moment
print(idx,row)

I modify the script slightly below, just to get the TOTAL OF ROW, I seriously think there is a better way, but for now, it is a hack.

Maybe the file also needs to be closed to prevent memory leaks. But I leave that for later.

Remember that you need to be inside Blender Scripting layout and place the script inside the Text Editor Panel. You then run the script using hotkey Alt+P.




When run, that Python code will import (open) the data.csv and go through the loop and print every row of data of the CSV file and print out the: ROW INDEX + the value for each row as LIST of STRING.

If your script is executed correctly, you will see something like below on the Blender console window:


This means that you successfully import the CSV data (Strings) into Blender.

Next step will be to process the data further. The data itself is really simple and ready to use, it is a type of LIST of STRING that you can simply use in Blender.

In order to have much simpler LIST data that I can process in loop for every line, I modify the script again like below:

import bpy # THIS IS A MUST TO IMPORT ALL BLENDER PYTHON MODULE
import csv # IMPORT CSV MODULE

# READ INPUT FILE

file = csv.reader(open('\\Users\Jimmy\Desktop\data.csv', newline=''), delimiter=',')

curRow = [] #empty placeholder for current row

for idx, row in enumerate(file):
if idx<11: # this is hard coded at the moment

curRow = row



The "curRow" variable there is a new LIST data variable that further break the CSV data into easier data to pass on.

005. STRING INTO 3D TEXT OBJECT
First data is String type and we can simply pass it to Blender to create 3D Text Object.

Via Google Search, I stumbled into this video tutorial: 

That video should explain clearly how you can use Python inside Blender to create Text Object with string data that you are supplying. 

So based from that information, our code becomes like below:

import bpy # THIS IS A MUST TO IMPORT ALL BLENDER PYTHON MODULE
import csv # IMPORT CSV MODULE

# READ INPUT FILE

file = csv.reader(open('\\Users\Jimmy\Desktop\data.csv', newline=''), delimiter=',')

curRow = [] #empty placeholder for current row

for idx, row in enumerate(file):
if idx<11: # this is hard coded but you can use

curRow = row

# display first data index as string
bpy.ops.object.text_add(location=(0,idx,0), rotation=(0,0,0))
bpy.ops.object.editmode_toggle()
bpy.ops.font.delete()
bpy.ops.font.text_insert(text=curRow[0])
bpy.ops.object.editmode_toggle()




You see how the first column of data which were originally in tabular format is now spit out as 3D Text Object and visualized in 3D!

We are good and on the right track!

The process continues for the other data on Column 2 and 3 (from the original Tabular Data)

006. NUMBER
Our number data is simple.

It is totally up to you how you like to visualize it. To be really simple, we just pass on the NUMBER into SCALE. I use 3D Polygon Cube and Scale them in one axis to create BAR kind of type of visualization, based on the number provided. And I simply place them on location.

The script is now like this:

import bpy # THIS IS A MUST TO IMPORT ALL BLENDER PYTHON MODULE
import csv # IMPORT CSV MODULE

# READ INPUT FILE

file = csv.reader(open('\\Users\Jimmy\Desktop\data.csv', newline=''), delimiter=',')

curRow = [] #empty placeholder for current row

for idx, row in enumerate(file):
if idx<11: # this is hard coded but you can use

curRow = row

# PROCESS COLUMN ONE
# display first data index as string
bpy.ops.object.text_add(location=(0,idx,0), rotation=(0,0,0))
bpy.ops.object.editmode_toggle()
bpy.ops.font.delete()
bpy.ops.font.text_insert(text=curRow[0])
bpy.ops.object.editmode_toggle()

# PROCESS COLUMN TWO
# create cube and transform resize
bpy.ops.mesh.primitive_cube_add(view_align=False, enter_editmode=False, location=(0, 0, 0), rotation=(0, 0, 0))


# store the location of current 3d cursor
saved_location = bpy.context.scene.cursor_location.copy() # returns a copy of the vector

# give 3dcursor new coordinates
bpy.context.scene.cursor_location = (0.0,0.0,-1.0)

# set the origin on the current object to the 3dcursor location
bpy.ops.object.origin_set(type='ORIGIN_CURSOR')

# set 3dcursor location back to the stored location
bpy.context.scene.cursor_location = saved_location


myScale = int(curRow[1])


#bpy.ops.transform.resize(value=(0.5, 0.5, 0.5))
bpy.ops.transform.resize(value=(0.5, 0.5, myScale), constraint_axis=(False, False, False), constraint_orientation='GLOBAL', mirror=False, proportional='DISABLED', proportional_edit_falloff='SMOOTH', proportional_size=1, snap=False, snap_target='CLOSEST', snap_point=(0, 0, 0), snap_align=False, snap_normal=(0, 0, 0), texture_space=False, release_confirm=False)

bpy.ops.transform.translate(value=(5, idx, 0), constraint_axis=(False, False, False))


I basically take the data from Column Two, as now passed in by curRow[1] placeholder. It does that for every line. It is as if you are create Cube, move it into place, and scale it accordingly based on the data provided.


It actually works, but not optimized. You see how the BARS are really too long in Blender Unit. Perhaps later I can optimize the data range so that they can be "normalized" and scaled into manageable size.

I also did something with the 3D Cube Origin there using Python. Looks tricky, but really simple once understood. Read below blog post:

Dealing with Origin using Python

This is unrelated but interesting:

007. COLOR DATA
Ok, now we get to data in Column 3, which is COLOR type. Well, it is actually STRING data (text), where I specified: RED, GREEN, or BLUE.

To accomplish this, I actually brainstormed a bit. Rather than using "all Python solutions", I am thinking that maybe it is easier if you prepare some Materials named "red", "green", "blue" that can then be linked into the Object.

It would be interesting too, to actually try to create new material and change the Color Diffuse value using Python. I might add addendum that does that.

Typically Color data is in format of Vector, RGB (123, 122, 123). I have not looked into it yet.

Anyhow, the idea is really to prepare 3 Materials named "red", "green", "blue", and then using Python to apply it to the BARS.

Assigning Materials using Python
Assigning certain Materials (with specific name) to certain object in Blender is slightly tricky because you need some understanding of Blender Data Flow and Linking.

This forum post helps me a lot:

But this one also helps:

What would be interesting is to do also try this, importing Materials from other BLEND. Very cool, but you may want to do this later.

I borrow the code from ATOM (on Material creation), and plonked it in into our script. I think I need to digest this information a little further and try to explain what happening later. But for now, the idea is like this:

1. In Blender, you need to create Material Data for each Object. You cannot assign material if you don't first assign Material.
2. Once you have Material Data, as we know, Blender can have "Multiple Slots" per Material.
3. For our task here, we just need a single Material Data and change the value of Material Slot into the Material name that we specified.
4. Make sure that you have prepared the "red", "green", and "blue" Materials before running the script.

The code becoming like this:


import bpy


import csv





# READ CSV DATA FILE





file = csv.reader(open('\\Users\Jimmy\Desktop\data.csv', newline=''), delimiter=',')





curRow = [] #empty placeholder for current row





# ATOM


def returnMaterialByName(passedName):


    result = None


    for m in bpy.data.materials:


        if m.name == passedName:


            result = m


            break


    return result





def createNewMaterial (passedName):


    tempMat = bpy.data.materials.new(passedName)


    if tempMat != None:


        tempMat.diffuse_color = (0.5,0.6,0.7)


        tempMat.diffuse_shader = 'LAMBERT'


        tempMat.diffuse_intensity = 1.0


        tempMat.specular_color = (0.9,0.9,0.9)


        tempMat.specular_shader = 'COOKTORR'


        tempMat.specular_intensity = 0.5


        tempMat.alpha = 1.0


        tempMat.ambient = 0.3


        tempMat.emit = 0.2


    return tempMat





def aquireOrCreateMaterial(passedName):


    tempMat = returnMaterialByName(passedName)


    if tempMat == None:


        tempMat = createNewMaterial(passedName)


    return tempMat








for idx, row in enumerate(file):


    if idx < 10: #still hard coded, how to check the total line size or length


        print(idx,row)


        curRow = row


        


        # display first data index as string


        bpy.ops.object.text_add(location=(0,idx,0), rotation=(0,0,0))


        bpy.ops.object.editmode_toggle()


        bpy.ops.font.delete()


        bpy.ops.font.text_insert(text=curRow[0])


        bpy.ops.object.editmode_toggle()


               


        # create cube and transform resize


        bpy.ops.mesh.primitive_cube_add(view_align=False, enter_editmode=False, location=(0, 0, 0), rotation=(0, 0, 0))


             





        # store the location of current 3d cursor


        saved_location = bpy.context.scene.cursor_location.copy()  # returns a copy of the vector


        


        # give 3dcursor new coordinates


        bpy.context.scene.cursor_location = (0.0,0.0,-1.0)


        


        # set the origin on the current object to the 3dcursor location


        bpy.ops.object.origin_set(type='ORIGIN_CURSOR')


        


        # set 3dcursor location back to the stored location


        bpy.context.scene.cursor_location = saved_location


        myScale = int(curRow[1])




        #bpy.ops.transform.resize(value=(0.5, 0.5, 0.5))


        bpy.ops.transform.resize(value=(0.5, 0.5, myScale), constraint_axis=(False, False, False), constraint_orientation='GLOBAL', mirror=False, proportional='DISABLED', proportional_edit_falloff='SMOOTH', proportional_size=1, snap=False, snap_target='CLOSEST', snap_point=(0, 0, 0), snap_align=False, snap_normal=(0, 0, 0), texture_space=False, release_confirm=False)





       


        bpy.ops.transform.translate(value=(5, idx, 0), constraint_axis=(False, False, False))







# create material


#bpy.ops.material.new()





# to script and create specific material with certain name will be difficult, another way is to create manually or import from outside BLEND and apply it using script


# if we manually already create certain material with certain name it will be easier, we simply need to apply it per selected object




        myObj = bpy.context.active_object


        print(myObj)


        


        


        # Program begins here.


        #bpy.ops.mesh.primitive_cylinder_add()


        #ob = bpy.context.active_object


        #ob.name = "myCylinder"


        me = myObj.data


        mat = aquireOrCreateMaterial ("myNewMaterial")


        me.materials.append(mat)





                


        # before applying below, you need to create material data slot first!


        myObj.material_slots[0].material = bpy.data.materials[curRow[2]]



When the script is run, hopefully you get the result like below:


So, there you have it, a whole documentation on how I accomplish this task of importing CSV data, passing it into Blender 3D Objects to visualize the data, using Python.

WHAT NEXT?
Actually even if the code is working, you can then take it further:
  • Further optimize the code and function
  • See what variations you can create with it
  • Create User Interface (UI) if required
This is a very basic example of how you can bring code from CSV, but I think you can do a lot from here.

Actually, this example is probably too hard for beginner programmers. It is probably useful to look into Python LIST as well, and how to work with list.

There are some parts where I jump and use the code without 100% understanding it. It does the job, but I need to take steps back and try to understand that part and simplify it.

The Material creation and assignment using Python should not be that complicated, just need a way to think and understand it.

At least if you and me manage to go through this exercise, you understand the workflow on how:
1. To bring data from CSV using Python in Blender.
2. Parse the data line by line using Python in Blender..
3. Further process the data and pass on the value to create 3D objects in Blender.

My WISHLIST to do using Python in near future:
  • Import and Export of Particle Data
  • Understand Python better, especially within Blender context, we want to be able to write out Blender data and read it back in on other 3D sofware like Maya, Houdini, Softimage.
  • Create Simulations
  • Create Random Looking Creature
  • Asset Management Pipeline
There is no doubt that Blender Python can be utilized to the level like below:
http://www.molecularmovies.com/toolkit/

UPDATE 2012.12.12: Blender PDB Atomic Molecular data importer
Well, in Blender 2.65 update, we have Protein Data Bank (PDB) Molecule Importer built in inside Blender!!! How awesome is that? You can actually study the Python of addons.
http://wiki.blender.org/index.php/Extensions:2.6/Py/Scripts/Import-Export/PDB

SEE ALSO: Bio Blender by Mike Pan
http://bioblender.eu/

This is Alcohol Atom formation.
Above is Alcohol PDB taken from here:
http://wbiomed.curtin.edu.au/biochem/tutorials/pdb/index.html

CHALLENGE:
For you, Blender Python genius that is mentally challenged:
  • Get live Twitter data and pass it into Blender
  • Get Facebook Data and visualize your friend's birthday across 12 months
  • Get live Yahoo! Weather data and visualize temperature of certain area in globe.

Post a Comment

MKRdezign

Contact Form

Name

Email *

Message *

Powered by Blogger.
Javascript DisablePlease Enable Javascript To See All Widget