Synchronisation in PVRVk#

The Vulkan API, and therefore PVRVk, has a detailed synchronisation scheme.

There are three important synchronisation objects: the semaphore, the fence and the event.

  • The semaphore is responsible for coarse-grained syncing of GPU operations, usually between queue submissions and/or presentation.

  • The fence is required to wait on GPU events on the CPU such as submissions, presentation image acquisitions, and some other cases.

  • The event is used for fine-grained control of the GPU from either the CPU or the GPU. It can also be used as part of layout transitions and dependencies.

Semaphores#

A command buffer imposes some order on submissions. When a command buffer is submitted to a queue, the Vulkan API allows considerable freedom to determine when the command buffer’s commands will actually be executed. This is either in relation to other command buffers submitted before or after it, or compared to other command buffers submitted together in the same batch (queue submission). The typical way to order these submissions is with semaphores.

Without using semaphores, the only guarantee imposed by Vulkan is that when two command buffer submissions happen, the commands of the second submission will not finish executing before the commands in the first submission have begun. This is not a strong guarantee.

The basic use of a semaphore is as follows:

  1. Create a semaphore.

  2. Add it to the SignalSemaphores list of the command buffer submission that needs to execute first.

  3. Add it in the WaitSemaphores list of the command buffer that is to be executed second.

  4. Set the Source Mask as the operations of the first command buffer that must be completed before the Destination Mask operations of the second command buffer begin. The more precise these are, the more overlap is allowed. Avoid blanket wait-for-everything statements, as this can impose considerable overhead by starving the GPU.

For example, by adding a semaphore with FragmentShader as the source and GeometryShader as the destination, this ensures that the FragmentShader of the first will have finished before the GeometryShader of the second begins. This implies that, for example, the VertexShaders might be executed simultaneously, or even in reverse order.

The above needs to be done whenever there are multiple command buffer submissions. In any case, it is usually recommended to do one big submission per frame whenever practical. The same semaphore can be used on different sides of the same command buffer submission - for instance in the wait list of one command buffer and the signal list of another, in the same queue submission. This is quite useful for reducing the number of queue submissions.

Fences#

Fences are simple: insert a fence on a supported operation whenever it is necessary to know (wait) on the CPU side when the operation is done. These operations are usually a command buffer submission or acquiring the next back buffer image.

This means that if a fence is waited on, any GPU commands that are submitted with the fence, and any commands dependent on them, are guaranteed to be over and done with.

Events#

Events can be used for fine-grained control, where a specific point in time during a command buffer execution needs to wait for either a CPU or a GPU side event. This can allow very precise threading of CPU and GPU side operations, but can become really complicated fast.

It is important to remember that events can be waited on in a command buffer with the waitEvents() function. They can be signalled both by the CPU by calling event->signal(), or when a specific command buffer point is reached by calling commandBuffer->signalEvent(). Execution of a specific point in a command buffer can be controlled either from the CPU side, or from the GPU with another command buffer submission.