Category

Programming

Increase performance for mobile VR

By | Programming, Trial By Error | No Comments

Making sure your application is optimized for VR is much more important than non-VR projects. This is because VR requires 60 – 90 FPS in order to give users a natural experience. If you fall below the 60FPS target than most likely users will start to feel sick.

Optimization

Optimizing your game or experience can vary depending on what is required as an end result. If you are able to make an experience that doesn’t need to be photorealistic, then changing from deferred rendering to forward rendering will save you a bunch of GPU cost. This is a quick cheap way of gaining performance but isn’t ideal if you want to achieve photo realism. Mainly due to the fact that forward rendering uses per-pixel lit mode for the brightest lights then uses per-vertex for up to 4 additional point lights. Every other light is computed as Spherical Harmonics (SH) which are just an approximation. If your goal is to achieve photo realism you will want to go with deferred shading which treats every light source per-pixel.

Baking

Baking is a good way to improve performance in your scene while maintaining the visual quality you need. In order to do this bake out the lighting data in your scene by marking objects as static and enabling global illumination in the enlighten engine. Keep in mind that when an object is marked static it cannot be affected by the physics pipeline. If you are required to have objects which move around in the scene then you will need to ensure that you have all your light probe volumes setup correctly to ensure the indirect lighting affects your dynamic objects.

Baking reflection probes is also a quick way to gain performance. As long as your reflective surfaces don’t need any real-time data (i.e. dynamic object reflections) you should almost always bake this out.

Draw Calls

To reduce the number of draw calls, simply start marking objects which share the same materials as static. This will allow the renderer to batch like materials and reduce the number of draw calls. It is better to use a texture atlas if you have multiple small textures and combine them to a shared material.

 

Occlusion Culling

We now need to start reducing the number of polygons rendered at any given time. Mobile VR is not very friendly when it comes to the number of polygons it can process, and based on today’s technology it is preferred to keep the poly count below 50k. In order to achieve this the first and most important place to start is to get back with your artist and re topologize any high poly art assets and optimize at the mesh level as much as you can. Now once you have your scene set up the way you want we need to enable occlusion culling. This process disables any geometry that is occluded by any other geometry at runtime. In order to see objects which are overdrawn look at the overdraw scene and start marking objects as occluder static, go to the occlusion culling window and make out the occlusion data.

 

Compression

Compress your art assets to the smallest size possible while trying to maintain the quality needed to pull off your scene. I know this kinda goes without saying but you’ll be surprised at how much memory you will save at the cost of minor quality degradation. This obviously depends on how many textures you have in a given scene but, every bit of performance gain counts.

Marshalling your data for Socket/IO? Maybe not.

By | Programming | No Comments

Hi again!  Today I’ve been working on send variable sized data packets over from my client to server using the simple TCP components I mentioned in my TCP Socket Programming post.

I have a few different classes that I want to serialize to byte[]s and send over the wire.   After reading a few suggestions online, I decided to try Marshalling.

With marshaling, you can copy the bytes as they are stored in memory – which is exactly what I needed.

 

            //Serialize data

            int size = Marshal.SizeOf(data);

            byte[] dataArr = new byte[size];

            IntPtr ptr = Marshal.AllocHGlobal(size);

            Marshal.StructureToPtr(data, ptr, false);

            Marshal.Copy(ptr, dataArr, 0, size);

            Marshal.FreeHGlobal(ptr);

This copies the byte[] into dataArr.

Here’s Microsoft’s documentation

https://msdn.microsoft.com/en-us/library/system.runtime.interopservices.marshal(v=vs.110).aspx

 

The only caveat with Marshalling is that you have to explicitly define the size of each property in your class.  And as I mentioned earlier, I was sending variable sized data packets.  Sometimes the property of my class would be quite large (ie. 500Kb) and other times really small (ie. 128 bytes).

[StructLayout(LayoutKind.Sequential, Pack = 1,CharSet = CharSet.Ansi)]
    public class HtsSenderPacket
    {  
        [MarshalAs(UnmanagedType.ByValArray, SizeConst = 300000)]
        public byte[] ScreenCapture;
        public HtsSenderPacket()
        {
            ScreenCapture = new byte[300000];
        }
    }

So I defined my ScreenCapture property to be a very large byte[].  I didn’t fill it all most of the time, but  I still was forced to send 300000 bytes.  So marshalling isn’t really the way to go for this case.  However, if you do decide to go this route, be sure to include the attribute:

[MarshalAs(UnmanagedType.ByValArray, SizeConst = 300000)]

I forgot this at first and was only sending the address and not the value.

Since marshalling wasn’t going to work for me, I ended up writing my class where I include the size of the variable property.

public class HtsPacket

{  
        private byte[] _screenCaptureSize = new byte[32];
        private byte[] _screenCapture; //variable size

        private int CLASS_SIZE { get { return 32 + _screenCapture.Length; } }
        public byte[] ScreenCapture { get { return _screenCapture; } }

        public HtsPacket(){
        }

        public HtsPacket(byte[] screenCapture){
          _screenCaptureSize = HtsEncoding.GetBytes(screenCapture.Length);
          _screenCapture = screenCapture;
        }    

        public byte[] ToBytes(){ //Serializes class to bytes        
            byte[] arr = new byte[CLASS_SIZE];
            Array.Copy(_screenCaptureSize, 0, arr, 0, _screenCaptureSize.Length);
            Array.Copy(_screenCapture, 0, arr, 32, _screenCapture.Length);
            return arr;
        }
    }

TCP Socket Programming in C#

By | Programming | No Comments

The last couple of weeks, I had been researching socket programming for our VR Conference project. Unity Networking and Photon both have bandwidth limitations that is preventing us to sharing desktops and other assets, so we needed to write a custom solution.  I did what you are doing… googled.  Most of the examples out there are for small amounts of data.

Take a look at the example from Microsoft:

https://msdn.microsoft.com/en-us/library/w89fhyex(v=vs.110).aspx

From the synchronous client example:

You create a new connection (check)
// Create a TCP/IP  socket.
Socket sender = new Socket(AddressFamily.InterNetwork,
SocketType.Stream, ProtocolType.Tcp );

From <https://msdn.microsoft.com/en-us/library/kb5kfec7(v=vs.110).aspx>

Then encode and send your data
// Encode the data string into a byte array.
byte[] msg = Encoding.ASCII.GetBytes("This is a test<EOF>");

// Send the data through the socket.
int bytesSent = sender.Send(msg);

From <https://msdn.microsoft.com/en-us/library/kb5kfec7(v=vs.110).aspx>

Pretty simple, right?  You think you’re sending a simple packet of data to your server.  But under the hood, your data packet could be broken up into various sized TCP packets.
If this is all you want to send, the server will get it and this example works just fine.

But if you need to send more, you’ll need to allocate a set number bytes to let the server know the size of the data you are transmitting, then read your buffer until you have it all.


 private void ReadCallback(IAsyncResult result)

{

 Socket socket = (Socket) result.AsyncState;

 try

 {

 int bytesRecieved = socket.EndReceive(result);

 _applicationPacketSize = GetApplicationPacketSize(_listenerBuffer); //returns the value from the first 4 bytes

 if (_applicationPacketSize == 0)

 {

 _tcpClient.Client.BeginReceive(_listenerBuffer, _offset, _listenerBuffer.Length - _offset, SocketFlags.None, new AsyncCallback(ReadCallback), _tcpClient.Client);

 return;

 }

 _offset += bytesRecieved;

 OnLogOutput(">> Read {0} bytes of data... new offset: {1}, application packet size: {2}", bytesRecieved, _offset, _applicationPacketSize);

 //Keep reading data until we reach _applicationPacketSize

 //if (_offset < _applicationPacketSize) //keep going, fill buffer

 if (_offset >= _applicationPacketSize) //we got it all and it's complete, the next bytes will be for a new packet

 {

 _offset -= _applicationPacketSize; //start over

 try

 {

 int packetSize = GetApplicationPacketSize(arr);
 //remove the HTS Header bytes

 byte[] data = new byte[packetSize - DATA_OFFSET];
 Array.Copy(arr, DATA_OFFSET, data, 0, data.Length);
 //You can marshal your data to deserialize it to a class
 } 
catch 
{

 OnLogOutput("Unable to deserialize the packet");

 }

 if (_offset > _applicationPacketSize) // We have a completed packet, but we have more bytes for the next packet

 {

 byte[] tmp = new byte[MAX_BUFFER_SIZE];

 Array.Copy(_listenerBuffer, _applicationPacketSize, tmp, 0, _offset);

 _listenerBuffer = tmp;

 }

 }

 _tcpClient.Client.BeginReceive(_listenerBuffer, _offset, _listenerBuffer.Length - _offset, SocketFlags.None, new AsyncCallback(ReadCallback), _tcpClient.Client);

 } 
catch (ObjectDisposedException exs)  //Socket has been closed.
{

 Disconnect();

} 
catch (Exception ex)
 {

 OnLogOutput(">> Error: {0}", ex.Message);

 _tcpClient.Client.BeginReceive(_listenerBuffer, _offset, _listenerBuffer.Length - _offset, SocketFlags.None, new AsyncCallback(ReadCallback), _tcpClient.Client);

 return;

 }

} 

 

Hope this help with your socket programming.

Game Engine Architecture, 2nd Edition Overview Ch.1 part 2

By | Gaming News, Programming | No Comments

Runtime Engine Architecture (section 1.6)?

This is where it delve into the different layers of systems and libraries that comprise the game engine, in my case, Unity.

Here’s a link to Figure 1.15 from the book:

http://www.gameenginebook.com/figures.html

Target Hardware, Device Drivers, OS, 3rd Party SDKs

The bottom four layers are dependent on the platforms your game will be running on.  Unity supports multiple platforms – meaning that under the hood, they use the drivers/SDKs so you can deploy your game on any of the platforms.

Platform Independence Layer, Core Systems, Resources

There isn’t much documentation regarding these layers from Unity.  I did read that Unity uses PhysX by NVIDIA for collision and physics.   It uses OpenGL for graphics…  Everything is nicely wrapped up in the Unity API.

https://docs.unity3d.com/ScriptReference/index.html

Rendering Engine

In Unity, the rendering engine is broken out into:

Camera: Cameras are components that display what a player will see.  It’s an imaging rectangle floating in your game scene.

Particle Systems: Particle Systems simulate motion using a lot of small 2D images, ie. clouds, fire, liquid.

Meshes: 3D Meshes are the main graphics primitive in Unity.  Unity doesn’t have a built-in modeling tool, but it supports .FBX, .dae, .3DS, .dxf and .obj files.  You can also import files directly from tools like Maya, 3D Studio Max,  Blender, …

For more details:

https://docs.unity3d.com/Manual/class-Mesh.html

Textures:  Textures are images (or movie files) that sit over your mesh.  Think of it as a vinyl wrapper over your car.

https://docs.unity3d.com/Manual/class-TextureImporter.html

Shaders: Shaders are scripts that have the math behind calculating the where and the color of each pixel rendered to your camera.  Unity provides built-in shaders and also lets you create custom shaders.

Here’s a great example of how you can use shaders to manipulate the color and vertices of your texture:

https://docs.unity3d.com/Manual/SL-SurfaceShaderExamples.html

Lighting:  In order to calculate the shading on a 3D object, Unity needs to know the intensity, direction, and color of the light that hits it.

I will be adding links to my posts detailing each topic here:

  • Profiling and Debugging
  • Collision and Physics
  • Animation
  • IO Devices
  • Audio
  • Networking
  • Game Play (AI, Scripting…)

Friends Don’t Let Friends Use Interfaces on MonoBehaviour Objects

By | Programming, Trial By Error | No Comments

Interfaces are great and solve many problems in the world of programming, but recently I have come across a scenario where they cause more harm than good in Unity. I am currently developing a VR zombie shooter game which I will simply call ZNN VR since I have yet to decide on a final name yet. You can read up on the current state of the game here.

The Problem

Since we are targeting Vive and Oculus Touch for ZNN VR  we needed to somehow handle switching between the two. The most obvious answer is to detect which headset is being used and programmatically tell the game to use the appropriate game objects. However, everything external to that should be agnostic about which device we are using so we don’t complicate our code with unnecessary if conditions. Read More

Game Engine Architecture, 2nd Edition Overview Ch.1 part 1

By | Gaming News, Programming | No Comments

What’s Game Engine Architecture?

... and why is this book so important?

For anyone that wants to learn more about game development, AR/VR development,  or computer science in general; this book is your Bible.  I’m reading this book and learning Unity’s game engine at the same time.  Unity is a very powerful game engine and has pretty much everything you need to build awesome software without having to know the details of the game engine itself.  They provide a variety of tutorials and robust documentation.  However, I still believe that you need to understand how things work “under the hood” to get the most out of the engine.

My goal is to build a reference guide to better correlate Unity’s game engine with the topics of this book. Read More