rhi: Add the basic infrastructure for tessellation support

...but this will only be supported with Vulkan and OpenGL 4.0+ and
OpenGL ES 3.2+ for the time being.

Taking the Vulkan model as our standard, the situation is the
following:

- Vulkan is ok, qsb secretly accepts .tesc and .tese files as input
  already (plus QShader already has the necessary plumbing when it
  comes to enums and such) To switch the tessellation domain origin to
  bottom left we require Vulkan 1.1 (don't bother with
  VK_KHR_maintenance2 on top of 1.0 at this point since 1.1 or 1.2
  implementations should be common by now). The change is essential to
  allow the same evaluation shader to work with both OpenGL and
  Vulkan: this way we can use the same shader source, declaring the
  tessellation winding order as CCW, with both APIs.

- OpenGL 4.0 and OpenGL ES 3.2 (or ES 3.1 with the Android extension
  pack, but we won't bother with checking that for now) can be made
  working without much complications, though we need to be careful
  when it comes to gathering and setting uniforms so that we do not
  leave the new tessellation stages out. We will stick to the Vulkan
  model in the sense that the inner and outer tessellation levels must
  be specified from the control shader, and cannot be specified from
  the host side, even though OpenGL would allow this. (basically the
  same story as with point size in vertex shaders)

- D3D11 would be no problem API-wise, and we could likely implement
  the support for hull and domain shader stages in the backend, but
  SPIRV-Cross does not support translating tessellation shaders to
  HLSL.  Attempting to feed in a .tesc or .tese file to qsb with
  --hlsl specified will always fail. One issue here is how hull
  shaders are structured, with the patchconstantfunc attribute
  specifying a separate function computing the patch constant
  data. With GLSL there is a single entry point in the tessellation
  control shader, which then performs both the calculations on the
  control points as well as the constant data (such as, the inner and
  outer tessellation factors).  One option here is to inject
  handwritten HLSL shaders in the .qsb files using qsb's replace (-r)
  mode, but this is not exactly a viable universal solution.

- Metal uses a different tessellation pipeline involving compute
  shaders. This needs more investigation but probably not something we
  can prioritize in practice. SPIRV-Cross does support this,
  generating a compute shader for control and a (post-)vertex shader
  for evaluation, presumably in order to enable MoltenVK to function
  when it comes to tessellation, but it is not clear yet how usable
  this is for us.

Change-Id: Ic953c63850bda5bc912c7ac354425041b43157ef
Reviewed-by: Andy Nichols <andy.nichols@qt.io>

src/gui/rhi/qrhi.cpp

@@ -695,6 +695,22 @@ Q_LOGGING_CATEGORY(QRHI_LOG_INFO, "qt.rhi.general")
     QRhi::newTextureArray() is functional. Note that even when texture arrays
     are not supported, arrays of textures are still available as those are two
     independent features.
+
+    \value Tessellation Indicates that the tessellation control and evaluation
+    stages are supported. When reported as supported, the topology of a
+    QRhiGraphicsPipeline can be set to
+    \l{QRhiGraphicsPipeline::Patches}{Patches}, the number of control points
+    can be set via
+    \l{QRhiGraphicsPipeline::setPatchControlPointCount()}{setPatchControlPointCount()},
+    and shaders for tessellation control and evaluation can be specified in the
+    QRhiShaderStage list. \b{Tessellation is considered an experimental feature
+    in QRhi and can only be expected to be supported with Vulkan for the time
+    being}, assuming the Vulkan implementation reports it as supported at run
+    time. Tessellation shaders have portability issues between APIs (for
+    example, translating GLSL/SPIR-V to HLSL is problematic due to the way hull
+    shaders are structured, whereas Metal uses a somewhat different
+    tessellation pipeline than others), and therefore no guarantees can be
+    given for a universal solution for now.
  */
 
 /*!
@@ -1443,8 +1459,17 @@ QDebug operator<<(QDebug dbg, const QRhiVertexInputLayout &v)
     Specifies the type of the shader stage.
 
     \value Vertex Vertex stage
-    \value Fragment Fragment (pixel) stage
+
-    \value Compute Compute stage (this may not always be supported at run time)
+    \value TessellationControlStage Tessellation control (hull shader) stage.
+    Must be used only when the QRhi::Tessellation feature is supported.
+
+    \value TessellationEvaluationStage Tessellation evaluation (domain shader)
+    stage. Must be used only when the QRhi::Tessellation feature is supported.
+
+    \value Fragment Fragment (pixel shader) stage
+
+    \value Compute Compute stage. Must be used only when the QRhi::Compute
+    feature is supported.
  */
 
 /*!
@@ -3228,7 +3253,9 @@ void QRhiImplementation::updateLayoutDesc(QRhiShaderResourceBindings *srb)
     Flag values to indicate which stages the shader resource is visible in
 
     \value VertexStage Vertex stage
-    \value FragmentStage Fragment (pixel) stage
+    \value TessellationControlStage Tessellation control (hull shader) stage
+    \value TessellationEvaluationStage Tessellation evaluation (domain shader) stage
+    \value FragmentStage Fragment (pixel shader) stage
     \value ComputeStage Compute stage
  */
 
@@ -4120,6 +4147,9 @@ QDebug operator<<(QDebug dbg, const QRhiShaderResourceBindings &srb)
     \value Lines
     \value LineStrip
     \value Points
+
+    \value Patches (only available if QRhi::Tessellation is supported, and
+    requires the tessellation stages to be present in the pipeline)
  */
 
 /*!
@@ -7390,6 +7420,10 @@ QRhiPassResourceTracker::BufferStage QRhiPassResourceTracker::toPassTrackerBuffe
     // pick the earlier stage (as this is going to be dstAccessMask)
     if (stages.testFlag(QRhiShaderResourceBinding::VertexStage))
         return QRhiPassResourceTracker::BufVertexStage;
+    if (stages.testFlag(QRhiShaderResourceBinding::TessellationControlStage))
+        return QRhiPassResourceTracker::BufTCStage;
+    if (stages.testFlag(QRhiShaderResourceBinding::TessellationEvaluationStage))
+        return QRhiPassResourceTracker::BufTEStage;
     if (stages.testFlag(QRhiShaderResourceBinding::FragmentStage))
         return QRhiPassResourceTracker::BufFragmentStage;
     if (stages.testFlag(QRhiShaderResourceBinding::ComputeStage))
@@ -7404,6 +7438,10 @@ QRhiPassResourceTracker::TextureStage QRhiPassResourceTracker::toPassTrackerText
     // pick the earlier stage (as this is going to be dstAccessMask)
     if (stages.testFlag(QRhiShaderResourceBinding::VertexStage))
         return QRhiPassResourceTracker::TexVertexStage;
+    if (stages.testFlag(QRhiShaderResourceBinding::TessellationControlStage))
+        return QRhiPassResourceTracker::TexTCStage;
+    if (stages.testFlag(QRhiShaderResourceBinding::TessellationEvaluationStage))
+        return QRhiPassResourceTracker::TexTEStage;
     if (stages.testFlag(QRhiShaderResourceBinding::FragmentStage))
         return QRhiPassResourceTracker::TexFragmentStage;
     if (stages.testFlag(QRhiShaderResourceBinding::ComputeStage))

src/gui/rhi/qrhi_p.h

@@ -295,6 +295,8 @@ class Q_GUI_EXPORT QRhiShaderStage
 public:
     enum Type {
         Vertex,
+        TessellationControl,
+        TessellationEvaluation,
         Fragment,
         Compute
     };
@@ -347,8 +349,10 @@ public:
 
     enum StageFlag {
         VertexStage = 1 << 0,
-        FragmentStage = 1 << 1,
+        TessellationControlStage = 1 << 1,
-        ComputeStage = 1 << 2
+        TessellationEvaluationStage = 1 << 2,
+        FragmentStage = 1 << 3,
+        ComputeStage = 1 << 4
     };
     Q_DECLARE_FLAGS(StageFlags, StageFlag)
 
@@ -1126,7 +1130,8 @@ public:
         TriangleFan,
         Lines,
         LineStrip,
-        Points
+        Points,
+        Patches
     };
 
     enum CullMode {
@@ -1297,6 +1302,9 @@ public:
     QRhiRenderPassDescriptor *renderPassDescriptor() const { return m_renderPassDesc; }
     void setRenderPassDescriptor(QRhiRenderPassDescriptor *desc) { m_renderPassDesc = desc; }
 
+    int patchControlPointCount() const { return m_patchControlPointCount; }
+    void setPatchControlPointCount(int count) { m_patchControlPointCount = count; }
+
     virtual bool create() = 0;
 
 protected:
@@ -1318,6 +1326,7 @@ protected:
     float m_lineWidth = 1.0f;
     int m_depthBias = 0;
     float m_slopeScaledDepthBias = 0.0f;
+    int m_patchControlPointCount = 3;
     QVarLengthArray<QRhiShaderStage, 4> m_shaderStages;
     QRhiVertexInputLayout m_vertexInputLayout;
     QRhiShaderResourceBindings *m_shaderResourceBindings = nullptr;
@@ -1654,7 +1663,8 @@ public:
         RenderBufferImport,
         ThreeDimensionalTextures,
         RenderTo3DTextureSlice,
-        TextureArrays
+        TextureArrays,
+        Tessellation
     };
 
     enum BeginFrameFlag {

src/gui/rhi/qrhi_p_p.h

@@ -610,6 +610,8 @@ public:
     enum BufferStage {
         BufVertexInputStage,
         BufVertexStage,
+        BufTCStage,
+        BufTEStage,
         BufFragmentStage,
         BufComputeStage
     };
@@ -628,6 +630,8 @@ public:
 
     enum TextureStage {
         TexVertexStage,
+        TexTCStage,
+        TexTEStage,
         TexFragmentStage,
         TexColorOutputStage,
         TexDepthOutputStage,

src/gui/rhi/qrhid3d11.cpp

@@ -550,6 +550,8 @@ bool QRhiD3D11::isFeatureSupported(QRhi::Feature feature) const
         return true;
     case QRhi::TextureArrays:
         return true;
+    case QRhi::Tessellation:
+        return false;
     default:
         Q_UNREACHABLE();
         return false;

src/gui/rhi/qrhigles2.cpp

@@ -442,6 +442,22 @@ QT_BEGIN_NAMESPACE
 #define GL_MAX_VARYING_VECTORS            0x8DFC
 #endif
 
+#ifndef GL_TESS_CONTROL_SHADER
+#define GL_TESS_CONTROL_SHADER            0x8E88
+#endif
+
+#ifndef GL_TESS_EVALUATION_SHADER
+#define GL_TESS_EVALUATION_SHADER         0x8E87
+#endif
+
+#ifndef GL_PATCH_VERTICES
+#define GL_PATCH_VERTICES                 0x8E72
+#endif
+
+#ifndef GL_PATCHES
+#define GL_PATCHES                        0x000E
+#endif
+
 /*!
     Constructs a new QRhiGles2InitParams.
 
@@ -836,6 +852,11 @@ bool QRhiGles2::create(QRhi::Flags flags)
 
     caps.texture3D = caps.ctxMajor >= 3; // 3.0
 
+    if (caps.gles)
+        caps.tessellation = caps.ctxMajor > 3 || (caps.ctxMajor == 3 && caps.ctxMinor >= 2); // ES 3.2
+    else
+        caps.tessellation = caps.ctxMajor >= 4; // 4.0
+
     if (caps.ctxMajor >= 3) { // 3.0 or ES 3.0
         GLint maxArraySize = 0;
         f->glGetIntegerv(GL_MAX_ARRAY_TEXTURE_LAYERS, &maxArraySize);
@@ -1233,6 +1254,8 @@ bool QRhiGles2::isFeatureSupported(QRhi::Feature feature) const
         return caps.texture3D;
     case QRhi::TextureArrays:
         return caps.maxTextureArraySize > 0;
+    case QRhi::Tessellation:
+        return caps.tessellation;
     default:
         Q_UNREACHABLE();
         return false;
@@ -2335,6 +2358,8 @@ static inline GLenum toGlTopology(QRhiGraphicsPipeline::Topology t)
         return GL_LINE_STRIP;
     case QRhiGraphicsPipeline::Points:
         return GL_POINTS;
+    case QRhiGraphicsPipeline::Patches:
+        return GL_PATCHES;
     default:
         Q_UNREACHABLE();
         return GL_TRIANGLES;
@@ -3433,6 +3458,14 @@ void QRhiGles2::executeBindGraphicsPipeline(QGles2CommandBuffer *cbD, QGles2Grap
         }
     }
 
+    if (psD->m_topology == QRhiGraphicsPipeline::Patches) {
+        const int cpCount = psD->m_patchControlPointCount;
+        if (forceUpdate || cpCount != state.cpCount) {
+            state.cpCount = cpCount;
+            f->glPatchParameteri(GL_PATCH_VERTICES, qMax(1, cpCount));
+        }
+    }
+
     f->glUseProgram(psD->program);
 }
 
@@ -4172,6 +4205,10 @@ static inline GLenum toGlShaderType(QRhiShaderStage::Type type)
     switch (type) {
     case QRhiShaderStage::Vertex:
         return GL_VERTEX_SHADER;
+    case QRhiShaderStage::TessellationControl:
+        return GL_TESS_CONTROL_SHADER;
+    case QRhiShaderStage::TessellationEvaluation:
+        return GL_TESS_EVALUATION_SHADER;
     case QRhiShaderStage::Fragment:
         return GL_FRAGMENT_SHADER;
     case QRhiShaderStage::Compute:
@@ -5369,6 +5406,15 @@ void QGles2GraphicsPipeline::destroy()
     }
 }
 
+static inline bool isGraphicsStage(const QRhiShaderStage &shaderStage)
+{
+    const QRhiShaderStage::Type t = shaderStage.type();
+    return t == QRhiShaderStage::Vertex
+            || t == QRhiShaderStage::TessellationControl
+            || t == QRhiShaderStage::TessellationEvaluation
+            || t == QRhiShaderStage::Fragment;
+}
+
 bool QGles2GraphicsPipeline::create()
 {
     QRHI_RES_RHI(QRhiGles2);
@@ -5386,23 +5432,39 @@ bool QGles2GraphicsPipeline::create()
 
     program = rhiD->f->glCreateProgram();
 
-    QShaderDescription vsDesc;
+    enum {
-    QShader::SeparateToCombinedImageSamplerMappingList vsSamplerMappingList;
+        VtxIdx = 0,
-    QShaderDescription fsDesc;
+        TEIdx,
-    QShader::SeparateToCombinedImageSamplerMappingList fsSamplerMappingList;
+        TCIdx,
+        FragIdx,
+        LastIdx
+    };
+    const auto descIdxForStage = [](const QRhiShaderStage &shaderStage) {
+        switch (shaderStage.type()) {
+        case QRhiShaderStage::Vertex:
+            return VtxIdx;
+        case QRhiShaderStage::TessellationControl:
+            return TCIdx;
+        case QRhiShaderStage::TessellationEvaluation:
+            return TEIdx;
+        case QRhiShaderStage::Fragment:
+            return FragIdx;
+        default:
+            break;
+        }
+        Q_UNREACHABLE();
+        return VtxIdx;
+    };
+    QShaderDescription desc[LastIdx];
+    QShader::SeparateToCombinedImageSamplerMappingList samplerMappingList[LastIdx];
     for (const QRhiShaderStage &shaderStage : qAsConst(m_shaderStages)) {
-        QShader shader = shaderStage.shader();
+        if (isGraphicsStage(shaderStage)) {
-        QShaderVersion shaderVersion;
+            const int idx = descIdxForStage(shaderStage);
-        if (shaderStage.type() == QRhiShaderStage::Vertex) {
+            QShader shader = shaderStage.shader();
-            vsDesc = shader.description();
+            QShaderVersion shaderVersion;
+            desc[idx] = shader.description();
             if (!rhiD->shaderSource(shaderStage, &shaderVersion).isEmpty()) {
-                vsSamplerMappingList = shader.separateToCombinedImageSamplerMappingList(
+                samplerMappingList[idx] = shader.separateToCombinedImageSamplerMappingList(
-                            { QShader::GlslShader, shaderVersion, shaderStage.shaderVariant() });
-            }
-        } else if (shaderStage.type() == QRhiShaderStage::Fragment) {
-            fsDesc = shader.description();
-            if (!rhiD->shaderSource(shaderStage, &shaderVersion).isEmpty()) {
-                fsSamplerMappingList = shader.separateToCombinedImageSamplerMappingList(
                             { QShader::GlslShader, shaderVersion, shaderStage.shaderVariant() });
             }
         }
@@ -5412,27 +5474,24 @@ bool QGles2GraphicsPipeline::create()
     QRhiGles2::ProgramCacheResult cacheResult = rhiD->tryLoadFromDiskOrPipelineCache(m_shaderStages.constData(),
                                                                                      m_shaderStages.count(),
                                                                                      program,
-                                                                                     vsDesc.inputVariables(),
+                                                                                     desc[VtxIdx].inputVariables(),
                                                                                      &cacheKey);
     if (cacheResult == QRhiGles2::ProgramCacheError)
         return false;
 
     if (cacheResult == QRhiGles2::ProgramCacheMiss) {
         for (const QRhiShaderStage &shaderStage : qAsConst(m_shaderStages)) {
-            if (shaderStage.type() == QRhiShaderStage::Vertex) {
+            if (isGraphicsStage(shaderStage)) {
-                if (!rhiD->compileShader(program, shaderStage, nullptr))
-                    return false;
-            } else if (shaderStage.type() == QRhiShaderStage::Fragment) {
                 if (!rhiD->compileShader(program, shaderStage, nullptr))
                     return false;
             }
         }
 
         // important when GLSL <= 150 is used that does not have location qualifiers
-        for (const QShaderDescription::InOutVariable &inVar : vsDesc.inputVariables())
+        for (const QShaderDescription::InOutVariable &inVar : desc[VtxIdx].inputVariables())
             rhiD->f->glBindAttribLocation(program, GLuint(inVar.location), inVar.name);
 
-        rhiD->sanityCheckVertexFragmentInterface(vsDesc, fsDesc);
+        rhiD->sanityCheckVertexFragmentInterface(desc[VtxIdx], desc[FragIdx]);
 
         if (!rhiD->linkProgram(program))
             return false;
@@ -5460,11 +5519,17 @@ bool QGles2GraphicsPipeline::create()
     // present in both shaders.
     QSet<int> activeUniformLocations;
 
-    for (const QShaderDescription::UniformBlock &ub : vsDesc.uniformBlocks())
+    for (const QRhiShaderStage &shaderStage : qAsConst(m_shaderStages)) {
-        rhiD->gatherUniforms(program, ub, &activeUniformLocations, &uniforms);
+        if (isGraphicsStage(shaderStage)) {
-
+            const int idx = descIdxForStage(shaderStage);
-    for (const QShaderDescription::UniformBlock &ub : fsDesc.uniformBlocks())
+            for (const QShaderDescription::UniformBlock &ub : desc[idx].uniformBlocks())
-        rhiD->gatherUniforms(program, ub, &activeUniformLocations, &uniforms);
+                rhiD->gatherUniforms(program, ub, &activeUniformLocations, &uniforms);
+            for (const QShaderDescription::InOutVariable &v : desc[idx].combinedImageSamplers())
+                rhiD->gatherSamplers(program, v, &samplers);
+            for (const QShader::SeparateToCombinedImageSamplerMapping &mapping : samplerMappingList[idx])
+                rhiD->gatherGeneratedSamplers(program, mapping, &samplers);
+        }
+    }
 
     std::sort(uniforms.begin(), uniforms.end(),
               [](const QGles2UniformDescription &a, const QGles2UniformDescription &b)
@@ -5472,18 +5537,6 @@ bool QGles2GraphicsPipeline::create()
         return a.offset < b.offset;
     });
 
-    for (const QShaderDescription::InOutVariable &v : vsDesc.combinedImageSamplers())
-        rhiD->gatherSamplers(program, v, &samplers);
-
-    for (const QShader::SeparateToCombinedImageSamplerMapping &mapping : vsSamplerMappingList)
-        rhiD->gatherGeneratedSamplers(program, mapping, &samplers);
-
-    for (const QShaderDescription::InOutVariable &v : fsDesc.combinedImageSamplers())
-        rhiD->gatherSamplers(program, v, &samplers);
-
-    for (const QShader::SeparateToCombinedImageSamplerMapping &mapping : fsSamplerMappingList)
-        rhiD->gatherGeneratedSamplers(program, mapping, &samplers);
-
     memset(uniformState, 0, sizeof(uniformState));
 
     currentSrb = nullptr;

src/gui/rhi/qrhigles2_p_p.h

@@ -600,6 +600,7 @@ struct QGles2CommandBuffer : public QRhiCommandBuffer
         float polyOffsetFactor;
         float polyOffsetUnits;
         float lineWidth;
+        int cpCount;
         void reset() { valid = false; }
         struct {
             // not part of QRhiGraphicsPipeline but used by setGraphicsPipeline()
@@ -962,7 +963,8 @@ public:
               intAttributes(true),
               screenSpaceDerivatives(false),
               programBinary(false),
-              texture3D(false)
+              texture3D(false),
+              tessellation(false)
         { }
         int ctxMajor;
         int ctxMinor;
@@ -1011,6 +1013,7 @@ public:
         uint screenSpaceDerivatives : 1;
         uint programBinary : 1;
         uint texture3D : 1;
+        uint tessellation : 1;
     } caps;
     QGles2SwapChain *currentSwapChain = nullptr;
     QSet<GLint> supportedCompressedFormats;

src/gui/rhi/qrhimetal.mm

@@ -617,6 +617,8 @@ bool QRhiMetal::isFeatureSupported(QRhi::Feature feature) const
         return true;
     case QRhi::TextureArrays:
         return true;
+    case QRhi::Tessellation:
+        return false;
     default:
         Q_UNREACHABLE();
         return false;

src/gui/rhi/qrhivulkan.cpp

@@ -424,6 +424,8 @@ bool QRhiVulkan::create(QRhi::Flags flags)
 
     f = inst->functions();
 
+    caps.vulkan11OrHigher = inst->apiVersion() >= QVersionNumber(1, 1);
+
     rhiFlags = flags;
 
     QList<VkQueueFamilyProperties> queueFamilyProps;
@@ -623,6 +625,8 @@ bool QRhiVulkan::create(QRhi::Flags flags)
             features.wideLines = VK_TRUE;
         if (physDevFeatures.largePoints)
             features.largePoints = VK_TRUE;
+        if (physDevFeatures.tessellationShader)
+            features.tessellationShader = VK_TRUE;
         if (physDevFeatures.textureCompressionETC2)
             features.textureCompressionETC2 = VK_TRUE;
         if (physDevFeatures.textureCompressionASTC_LDR)
@@ -689,7 +693,9 @@ bool QRhiVulkan::create(QRhi::Flags flags)
 
     caps.wideLines = physDevFeatures.wideLines;
 
-    caps.texture3DSliceAs2D = inst->apiVersion() >= QVersionNumber(1, 1);
+    caps.texture3DSliceAs2D = caps.vulkan11OrHigher;
+
+    caps.tessellation = physDevFeatures.tessellationShader;
 
     if (!importedAllocator) {
         VmaVulkanFunctions afuncs;
@@ -3973,6 +3979,10 @@ static inline VkPipelineStageFlags toVkPipelineStage(QRhiPassResourceTracker::Bu
         return VK_PIPELINE_STAGE_VERTEX_INPUT_BIT;
     case QRhiPassResourceTracker::BufVertexStage:
         return VK_PIPELINE_STAGE_VERTEX_SHADER_BIT;
+    case QRhiPassResourceTracker::BufTCStage:
+        return VK_PIPELINE_STAGE_TESSELLATION_CONTROL_SHADER_BIT;
+    case QRhiPassResourceTracker::BufTEStage:
+        return VK_PIPELINE_STAGE_TESSELLATION_EVALUATION_SHADER_BIT;
     case QRhiPassResourceTracker::BufFragmentStage:
         return VK_PIPELINE_STAGE_FRAGMENT_SHADER_BIT;
     case QRhiPassResourceTracker::BufComputeStage:
@@ -4039,6 +4049,10 @@ static inline VkPipelineStageFlags toVkPipelineStage(QRhiPassResourceTracker::Te
     switch (stage) {
     case QRhiPassResourceTracker::TexVertexStage:
         return VK_PIPELINE_STAGE_VERTEX_SHADER_BIT;
+    case QRhiPassResourceTracker::TexTCStage:
+        return VK_PIPELINE_STAGE_TESSELLATION_CONTROL_SHADER_BIT;
+    case QRhiPassResourceTracker::TexTEStage:
+        return VK_PIPELINE_STAGE_TESSELLATION_EVALUATION_SHADER_BIT;
     case QRhiPassResourceTracker::TexFragmentStage:
         return VK_PIPELINE_STAGE_FRAGMENT_SHADER_BIT;
     case QRhiPassResourceTracker::TexColorOutputStage:
@@ -4301,6 +4315,8 @@ bool QRhiVulkan::isFeatureSupported(QRhi::Feature feature) const
         return caps.texture3DSliceAs2D;
     case QRhi::TextureArrays:
         return true;
+    case QRhi::Tessellation:
+        return caps.tessellation;
     default:
         Q_UNREACHABLE();
         return false;
@@ -5243,6 +5259,10 @@ static inline VkShaderStageFlagBits toVkShaderStage(QRhiShaderStage::Type type)
     switch (type) {
     case QRhiShaderStage::Vertex:
         return VK_SHADER_STAGE_VERTEX_BIT;
+    case QRhiShaderStage::TessellationControl:
+        return VK_SHADER_STAGE_TESSELLATION_CONTROL_BIT;
+    case QRhiShaderStage::TessellationEvaluation:
+        return VK_SHADER_STAGE_TESSELLATION_EVALUATION_BIT;
     case QRhiShaderStage::Fragment:
         return VK_SHADER_STAGE_FRAGMENT_BIT;
     case QRhiShaderStage::Compute:
@@ -5307,6 +5327,8 @@ static inline VkPrimitiveTopology toVkTopology(QRhiGraphicsPipeline::Topology t)
         return VK_PRIMITIVE_TOPOLOGY_LINE_STRIP;
     case QRhiGraphicsPipeline::Points:
         return VK_PRIMITIVE_TOPOLOGY_POINT_LIST;
+    case QRhiGraphicsPipeline::Patches:
+        return VK_PRIMITIVE_TOPOLOGY_PATCH_LIST;
     default:
         Q_UNREACHABLE();
         return VK_PRIMITIVE_TOPOLOGY_TRIANGLE_LIST;
@@ -5520,6 +5542,10 @@ static inline VkShaderStageFlags toVkShaderStageFlags(QRhiShaderResourceBinding:
         s |= VK_SHADER_STAGE_FRAGMENT_BIT;
     if (stage.testFlag(QRhiShaderResourceBinding::ComputeStage))
         s |= VK_SHADER_STAGE_COMPUTE_BIT;
+    if (stage.testFlag(QRhiShaderResourceBinding::TessellationControlStage))
+        s |= VK_SHADER_STAGE_TESSELLATION_CONTROL_BIT;
+    if (stage.testFlag(QRhiShaderResourceBinding::TessellationEvaluationStage))
+        s |= VK_SHADER_STAGE_TESSELLATION_EVALUATION_BIT;
     return VkShaderStageFlags(s);
 }
 
@@ -6939,6 +6965,36 @@ bool QVkGraphicsPipeline::create()
     inputAsmInfo.primitiveRestartEnable = (m_topology == TriangleStrip || m_topology == LineStrip);
     pipelineInfo.pInputAssemblyState = &inputAsmInfo;
 
+    VkPipelineTessellationStateCreateInfo tessInfo;
+#ifdef VK_VERSION_1_1
+    VkPipelineTessellationDomainOriginStateCreateInfo originInfo;
+#endif
+    if (m_topology == Patches) {
+        memset(&tessInfo, 0, sizeof(tessInfo));
+        tessInfo.sType = VK_STRUCTURE_TYPE_PIPELINE_TESSELLATION_STATE_CREATE_INFO;
+        tessInfo.patchControlPoints = uint32_t(qMax(1, m_patchControlPointCount));
+
+        // To be able to use the same tess.evaluation shader with both OpenGL
+        // and Vulkan, flip the tessellation domain origin to be lower left.
+        // This allows declaring the winding order in the shader to be CCW and
+        // still have it working with both APIs. This requires Vulkan 1.1 (or
+        // VK_KHR_maintenance2 but don't bother with that).
+#ifdef VK_VERSION_1_1
+        if (rhiD->caps.vulkan11OrHigher) {
+            memset(&originInfo, 0, sizeof(originInfo));
+            originInfo.sType = VK_STRUCTURE_TYPE_PIPELINE_TESSELLATION_DOMAIN_ORIGIN_STATE_CREATE_INFO;
+            originInfo.domainOrigin = VK_TESSELLATION_DOMAIN_ORIGIN_LOWER_LEFT;
+            tessInfo.pNext = &originInfo;
+        } else {
+            qWarning("Proper tessellation support requires Vulkan 1.1 or newer, leaving domain origin unset");
+        }
+#else
+        qWarning("QRhi was built without Vulkan 1.1 headers, this is not sufficient for proper tessellation support");
+#endif
+
+        pipelineInfo.pTessellationState = &tessInfo;
+    }
+
     VkPipelineRasterizationStateCreateInfo rastInfo;
     memset(&rastInfo, 0, sizeof(rastInfo));
     rastInfo.sType = VK_STRUCTURE_TYPE_PIPELINE_RASTERIZATION_STATE_CREATE_INFO;

src/gui/rhi/qrhivulkan_p_p.h

@@ -889,6 +889,8 @@ public:
         bool debugMarkers = false;
         bool vertexAttribDivisor = false;
         bool texture3DSliceAs2D = false;
+        bool tessellation = false;
+        bool vulkan11OrHigher = false;
     } caps;
 
     VkPipelineCache pipelineCache = VK_NULL_HANDLE;

tests/manual/rhi/CMakeLists.txt

@@ -23,3 +23,4 @@ add_subdirectory(instancing)
 add_subdirectory(noninstanced)
 add_subdirectory(tex3d)
 add_subdirectory(texturearray)
+add_subdirectory(tessellation)

tests/manual/rhi/tessellation/CMakeLists.txt

@@ -0,0 +1,22 @@
+qt_internal_add_manual_test(tessellation
+    GUI
+    SOURCES
+        tessellation.cpp
+    PUBLIC_LIBRARIES
+        Qt::Gui
+        Qt::GuiPrivate
+)
+
+set(tessellation_resource_files
+    "test.vert.qsb"
+    "test.tesc.qsb"
+    "test.tese.qsb"
+    "test.frag.qsb"
+)
+
+qt_internal_add_resource(tessellation "tessellation"
+    PREFIX
+        "/"
+    FILES
+        ${tessellation_resource_files}
+)

tests/manual/rhi/tessellation/buildshaders.bat

@@ -0,0 +1,4 @@
+qsb --glsl 320es,410 test.vert -o test.vert.qsb
+qsb --glsl 320es,410 test.tesc -o test.tesc.qsb
+qsb --glsl 320es,410 test.tese -o test.tese.qsb
+qsb --glsl 320es,410 test.frag -o test.frag.qsb

tests/manual/rhi/tessellation/tessellation.cpp

@@ -0,0 +1,155 @@
+/****************************************************************************
+**
+** Copyright (C) 2021 The Qt Company Ltd.
+** Contact: https://www.qt.io/licensing/
+**
+** This file is part of the examples of the Qt Toolkit.
+**
+** $QT_BEGIN_LICENSE:BSD$
+** Commercial License Usage
+** Licensees holding valid commercial Qt licenses may use this file in
+** accordance with the commercial license agreement provided with the
+** Software or, alternatively, in accordance with the terms contained in
+** a written agreement between you and The Qt Company. For licensing terms
+** and conditions see https://www.qt.io/terms-conditions. For further
+** information use the contact form at https://www.qt.io/contact-us.
+**
+** BSD License Usage
+** Alternatively, you may use this file under the terms of the BSD license
+** as follows:
+**
+** "Redistribution and use in source and binary forms, with or without
+** modification, are permitted provided that the following conditions are
+** met:
+**   * Redistributions of source code must retain the above copyright
+**     notice, this list of conditions and the following disclaimer.
+**   * Redistributions in binary form must reproduce the above copyright
+**     notice, this list of conditions and the following disclaimer in
+**     the documentation and/or other materials provided with the
+**     distribution.
+**   * Neither the name of The Qt Company Ltd nor the names of its
+**     contributors may be used to endorse or promote products derived
+**     from this software without specific prior written permission.
+**
+**
+** THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+** "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+** LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+** A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+** OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+** SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+** LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+** DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+** THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+** (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+** OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE."
+**
+** $QT_END_LICENSE$
+**
+****************************************************************************/
+
+#include "../shared/examplefw.h"
+
+static const float tri[] = {
+    0.0f, 0.5f, 0.0f,     0.0f, 0.0f, 1.0f,
+    -0.5f, -0.5f, 0.0f,   1.0f, 0.0f, 0.0f,
+    0.5f, -0.5f, 0.0f,    0.0f, 1.0f, 0.0f,
+};
+
+struct {
+    QVector<QRhiResource *> releasePool;
+    QRhiBuffer *vbuf = nullptr;
+    QRhiBuffer *ubuf = nullptr;
+    QRhiShaderResourceBindings *srb = nullptr;
+    QRhiGraphicsPipeline *ps = nullptr;
+    QRhiResourceUpdateBatch *initialUpdates = nullptr;
+    QMatrix4x4 winProj;
+    float time = 0.0f;
+} d;
+
+void Window::customInit()
+{
+    if (!m_r->isFeatureSupported(QRhi::Tessellation))
+        qFatal("Tessellation is not supported");
+
+    d.vbuf = m_r->newBuffer(QRhiBuffer::Immutable, QRhiBuffer::VertexBuffer, sizeof(tri));
+    d.vbuf->create();
+    d.releasePool << d.vbuf;
+
+    d.ubuf = m_r->newBuffer(QRhiBuffer::Dynamic, QRhiBuffer::UniformBuffer, 64 + 4 + 4);
+    d.ubuf->create();
+    d.releasePool << d.ubuf;
+
+    d.srb = m_r->newShaderResourceBindings();
+    d.releasePool << d.srb;
+    const QRhiShaderResourceBinding::StageFlags tese = QRhiShaderResourceBinding::TessellationEvaluationStage;
+    d.srb->setBindings({ QRhiShaderResourceBinding::uniformBuffer(0, tese, d.ubuf) });
+    d.srb->create();
+
+    d.ps = m_r->newGraphicsPipeline();
+    d.releasePool << d.ps;
+
+    d.ps->setTopology(QRhiGraphicsPipeline::Patches);
+    d.ps->setPatchControlPointCount(3);
+
+    d.ps->setShaderStages({
+        { QRhiShaderStage::Vertex, getShader(QLatin1String(":/test.vert.qsb")) },
+        { QRhiShaderStage::TessellationControl, getShader(QLatin1String(":/test.tesc.qsb")) },
+        { QRhiShaderStage::TessellationEvaluation, getShader(QLatin1String(":/test.tese.qsb")) },
+        { QRhiShaderStage::Fragment, getShader(QLatin1String(":/test.frag.qsb")) }
+    });
+
+    d.ps->setCullMode(QRhiGraphicsPipeline::Back);
+    d.ps->setDepthTest(true);
+    d.ps->setDepthWrite(true);
+    QRhiVertexInputLayout inputLayout;
+    inputLayout.setBindings({
+        { 6 * sizeof(float) }
+    });
+    inputLayout.setAttributes({
+        { 0, 0, QRhiVertexInputAttribute::Float3, 0 },
+        { 0, 1, QRhiVertexInputAttribute::Float3, 3 * sizeof(float) }
+    });
+    d.ps->setVertexInputLayout(inputLayout);
+    d.ps->setShaderResourceBindings(d.srb);
+    d.ps->setRenderPassDescriptor(m_rp);
+    d.ps->create();
+
+    d.initialUpdates = m_r->nextResourceUpdateBatch();
+    d.initialUpdates->uploadStaticBuffer(d.vbuf, tri);
+    const float amplitude = 0.5f;
+    d.initialUpdates->updateDynamicBuffer(d.ubuf, 68, 4, &amplitude);
+}
+
+void Window::customRelease()
+{
+    qDeleteAll(d.releasePool);
+    d.releasePool.clear();
+}
+
+void Window::customRender()
+{
+    const QSize outputSizeInPixels = m_sc->currentPixelSize();
+    QRhiCommandBuffer *cb = m_sc->currentFrameCommandBuffer();
+    QRhiResourceUpdateBatch *u = m_r->nextResourceUpdateBatch();
+    if (d.initialUpdates) {
+        u->merge(d.initialUpdates);
+        d.initialUpdates->release();
+        d.initialUpdates = nullptr;
+    }
+    if (d.winProj != m_proj) {
+        d.winProj = m_proj;
+        u->updateDynamicBuffer(d.ubuf, 0, 64, d.winProj.constData());
+    }
+    u->updateDynamicBuffer(d.ubuf, 64, 4, &d.time);
+    d.time += 0.1f;
+
+    cb->beginPass(m_sc->currentFrameRenderTarget(), m_clearColor, { 1.0f, 0 }, u);
+    cb->setGraphicsPipeline(d.ps);
+    cb->setViewport({ 0, 0, float(outputSizeInPixels.width()), float(outputSizeInPixels.height()) });
+    cb->setShaderResources();
+    QRhiCommandBuffer::VertexInput vbufBinding(d.vbuf, 0);
+    cb->setVertexInput(0, 1, &vbufBinding);
+    cb->draw(3);
+    cb->endPass();
+}

tests/manual/rhi/tessellation/test.frag

@@ -0,0 +1,10 @@
+#version 440
+
+layout(location = 0) in vec3 v_color;
+
+layout(location = 0) out vec4 fragColor;
+
+void main()
+{
+    fragColor = vec4(v_color, 1.0);
+}

tests/manual/rhi/tessellation/test.tesc

@@ -0,0 +1,21 @@
+#version 440
+
+layout(vertices = 3) out;
+
+layout(location = 0) in vec3 inColor[];
+
+layout(location = 0) out vec3 outColor[];
+
+void main()
+{
+    if (gl_InvocationID == 0) {
+        gl_TessLevelOuter[0] = 4.0;
+        gl_TessLevelOuter[1] = 4.0;
+        gl_TessLevelOuter[2] = 4.0;
+
+        gl_TessLevelInner[0] = 4.0;
+    }
+
+    gl_out[gl_InvocationID].gl_Position = gl_in[gl_InvocationID].gl_Position;
+    outColor[gl_InvocationID] = inColor[gl_InvocationID];
+}

tests/manual/rhi/tessellation/test.tese

@@ -0,0 +1,20 @@
+#version 440
+
+layout(triangles, fractional_odd_spacing, ccw) in;
+
+layout(location = 0) in vec3 inColor[];
+
+layout(location = 0) out vec3 outColor;
+
+layout(std140, binding = 0) uniform buf {
+    mat4 mvp;
+    float time;
+    float amplitude;
+};
+
+void main()
+{
+    gl_Position = mvp * ((gl_TessCoord.x * gl_in[0].gl_Position) + (gl_TessCoord.y * gl_in[1].gl_Position) + (gl_TessCoord.z * gl_in[2].gl_Position));
+    gl_Position.x += sin(time + gl_Position.y) * amplitude;
+    outColor = gl_TessCoord.x * inColor[0] + gl_TessCoord.y * inColor[1] + gl_TessCoord.z * inColor[2];
+}

tests/manual/rhi/tessellation/test.vert

@@ -0,0 +1,12 @@
+#version 440
+
+layout(location = 0) in vec3 position;
+layout(location = 1) in vec3 color;
+
+layout(location = 0) out vec3 v_color;
+
+void main()
+{
+    gl_Position = vec4(position, 1.0);
+    v_color = color;
+}