Rendering related API - any thread safety?

[Bat2K]

Hi,

I have a question on thread safety - basically what API part are thread-safe and what are not?
I saw demos utilizing gh_utils.exe_script for socket server, texture preloading and plain calculations.

More concrete example is offloading screenshot making into separate thread in parallel with the rendering - my interest is more of understanding in this case rather than optimization. For now my naive result is - spawning python thread and calling do_screenshot inside gives me segfault (context is living in different thread - that's the point i guess).

So, whether I can use some rendering related api in a different thread or not? I guess for that i need a shared opengl context and some mechanism to control it...

Thanks!

[JeGX]

There are currently three functions for launching a script in a system thread:

Code: [Select]

gh_utils.execute_script(script_name, threaded)
gh_utils.execute_script_v2(script_name, threaded, create_worker_wind3d)
gh_utils.execute_script_v3(script_name, threaded, create_worker_wind3d, affinity_mask)

where:
- script_name : string, name of a ZOMBIE script
- threaded: 0 or 1 to start or not the script in a system thread
- create_worker_wind3d: 0 or 1 to create or not an invisible window with an OpenGL context (shared)
- affinity_mask: thread affinity mask (for windows only)

So if you want to create OpenGL textures in a threaded script:

Code: [Select]

gh_utils.execute_script_v2("resource_loading", 1, 1)

In this thread script, you can use all Lua/Python functions. All gh_renderer / ghtexture, gh_gpu_program functions will work but they will use a new renderer that has a rendering context shared with the main renderer.

Now an important thing in Lua (and maybe in Python too, I can't remenber, you have to test to be sure): each Lua script in a separate thread has its own virtual machine context and can not share variables with other scripts. In Lua there is no lock, each script can run at full speed in its thread. This is possible only because each script lives in its own world.

To share data between scripts, GeeXLab has a special API for that: gh_utils.shared_variable_xxxxx(). Thanks to this API, you can create special variables that are visible from any scripts in read/write. You can even communicate between a Lua and a Python script.

Now for the thread safety side... it's up to you to manage threading issues. If you need it I can add a function that creates a thread lock (via a mutex on Linux).

Managing threading it not easy (and GeeXLab has only a basic support of threading) so do your tests and let me know if you have weird crashes.

I will try to look at the do_screenshot() issue when the function is called from another thread.



 

[JeGX]

I think you can't call do screenshot from another thread. Because there is no way to read the framebuffer of the main GL context from another thread.  Maybe a possible optimization is to put the image saving in a separate thread. So gh_utils.do_screenshot() is called in the main FRAME script but image saving is done is a new thread. Maybe like that, doing screenshots will be faster. I have to add this support in GeeXLab.

[Bat2K]

JeGX,

thanks for clarifying on the topic. Regarding do_screenshot in a separate thread as I mentioned I don't have an optimization interest, it was just an experiment followed by the questions on how the GXL works in more details. Your advice on doing io in the separate thread is expected and appreciated

Thanks!

[JeGX]

For the sake of curiosity, I spent some time yesterday on improving screenshot speed and I ended up on a X10 speed boost thanks to a simple technique: the image saving is done in a separate thread. Here are framerates with my test demo:

- normal running, without screenshots: 1800 FPS
- running with screenshots: 30 FPS
- running with new screenshots function: 300 FPS

Will be available in GeeXLab 0.29.3+

[Bat2K]

I'm glad i triggered an improvement Thanks!