Video driver enablement changes for Shikra#957
Open
dikshita-agarwal wants to merge 36 commits intoqualcomm-linux:early/hwe/shikra/driversfrom
Open
Video driver enablement changes for Shikra#957dikshita-agarwal wants to merge 36 commits intoqualcomm-linux:early/hwe/shikra/driversfrom
dikshita-agarwal wants to merge 36 commits intoqualcomm-linux:early/hwe/shikra/driversfrom
Conversation
…o codec binding Kaanapali SOC brings in the new generation of video IP i.e iris4. When compared to previous generation, iris3x, it has, - separate power domains for stream and pixel processing hardware blocks (bse and vpp). - additional power domain for apv codec. - power domains for individual pipes (VPPx). - different clocks and reset lines. Iommus include all the different stream-ids which can be possibly generated by vpu4 video hardware in both secure and non secure execution mode. The vpu have reserved iova, i.e some portion of the addressable range is reserved for IO registers. Iris_resv would describe the acceptable address range. Link: https://lore.kernel.org/linux-media/20251017-knp_video-v2-1-f568ce1a4be3@oss.qualcomm.com/ Signed-off-by: Vikash Garodia <vikash.garodia@oss.qualcomm.com>
Add support for the kaanapali platform by re-using the SM8550 definitions and using the vpu4 ops. Move the configurations that differs in a per-SoC platform header, that will contain SoC specific data. Link: https://lore.kernel.org/all/20251017-knp_video-v2-8-f568ce1a4be3@oss.qualcomm.com/ Co-developed-by: Vishnu Reddy <quic_bvisredd@quicinc.com> Signed-off-by: Vishnu Reddy <quic_bvisredd@quicinc.com> Signed-off-by: Vikash Garodia <vikash.garodia@oss.qualcomm.com> Reviewed-by: Bryan O'Donoghue <bryan.odonoghue@linaro.org>
…y Linux Most Qualcomm platforms feature a inhouse hypervisor (such as Gunyah or QHEE), which typically handles IOMMU configuration. This includes mapping memory regions and device memory resources for remote processors by intercepting qcom_scm_pas_auth_and_reset() calls. These mappings are later removed during teardown. Additionally, SHM bridge setup is required to enable memory protection for both remoteproc metadata and its memory regions. When the hypervisor is absent, the operating system must perform these configurations instead. Support for handling IOMMU and SHM setup in the absence of a hypervisor is now in place. Extend the Iris driver to enable this functionality on platforms where IOMMU is managed by Linux (i.e., non-Gunyah, non-QHEE). Additionally, the Iris driver must map the firmware and its required resources to the firmware SID, which is now specified via the device tree. Link: https://lore.kernel.org/lkml/20250819165447.4149674-12-mukesh.ojha@oss.qualcomm.com/ Signed-off-by: Vikash Garodia <vikash.garodia@oss.qualcomm.com> Signed-off-by: Mukesh Ojha <mukesh.ojha@oss.qualcomm.com>
The Qualcomm MDT loader changed qcom_mdt_pas_load() to take only four arguments and use the PAS context for relocation and memory handling. Update the iris firmware loading path to match that interface by dropping the temporary memremap/memunmap flow, removing the extra mem_virt argument from qcom_mdt_pas_load(), and simplifying the error path accordingly. This keeps the iris driver aligned with the SCM/PAS loader changes and fixes the build failure caused by the old five-argument call. Signed-off-by: Gourav Kumar <gouravk@qti.qualcomm.com>
…culation get_framesize_raw_nv12() currently adds SZ_4K to the UV plane size and an additional SZ_8K to the total buffer size. This inflates the calculated sizeimage and leads userspace to over-allocate buffers without a clear requirement. Remove the extra SZ_4K/SZ_8K padding and compute the NV12 size as the sum of Y and UV planes, keeping the final ALIGN(size, SZ_4K) intact. Link: https://lore.kernel.org/linux-arm-msm/20260331-fix_venus_bug_issue-v1-1-e4ae7a1d8db2@oss.qualcomm.com Fixes: e1cb72d ("media: venus: helpers: move frame size calculations on common place") Signed-off-by: Renjiang Han <renjiang.han@oss.qualcomm.com>
…s on v4 Encoder HFI capabilities on v4 advertise a 16-pixel step for frame and blur dimensions. This is overly restrictive and can cause userspace caps negotiation to fail even for valid resolutions. Relax the advertised step size to 1 and keep alignment enforcement in buffer layout and size calculations. Link: https://lore.kernel.org/linux-arm-msm/20260331-fix_venus_bug_issue-v1-2-e4ae7a1d8db2@oss.qualcomm.com Fixes: 8b88cab ("media: venus: hfi_plat_v4: Populate codecs and capabilities for v4") Signed-off-by: Renjiang Han <renjiang.han@oss.qualcomm.com>
Encoder HFI capabilities on v6 enforce a 16-pixel step for frame and blur dimensions, which does not reflect actual hardware requirements and can reject valid userspace configurations. Relax the step size to 1 while leaving min/max limits unchanged. Link: https://lore.kernel.org/linux-arm-msm/20260331-fix_venus_bug_issue-v1-3-e4ae7a1d8db2@oss.qualcomm.com Fixes: 869d77e ("media: venus: hfi_plat_v6: Populate capabilities for v6") Signed-off-by: Renjiang Han <renjiang.han@oss.qualcomm.com>
…coding on SC7280 Follow commit bfe1326 ("venus: Fix for H265 decoding failure.") and increase H265D_MAX_SLICE following firmware requirements on that platform. Otherwise decoding of the H.265 streams fails withthe insufficient scratch_1 buffer size from the firmware. Link: https://lore.kernel.org/all/20260327-venus-iris-flip-switch-v5-3-2f4b6c636927@oss.qualcomm.com/ Signed-off-by: Dmitry Baryshkov <dmitry.baryshkov@oss.qualcomm.com> Reviewed-by: Dikshita Agarwal <dikshita.agarwal@oss.qualcomm.com> Reviewed-by: Vikash Garodia <vikash.garodia@oss.qualcomm.com> Reviewed-by: Konrad Dybcio <konrad.dybcio@oss.qualcomm.com>
With the Iris and Venus driver having more or less feature parity for HFI 6xx platforms and with Iris gaining support for SC7280, flip the switch. Use Iris by default for SM8250 and SC7280, the platforms which are supported by both drivers, and use Venus only if Iris is not compiled at all. Use IS_ENABLED to strip out the code and data structures which are used by the disabled platforms. Link: https://lore.kernel.org/all/20260327-venus-iris-flip-switch-v5-4-2f4b6c636927@oss.qualcomm.com/ Reviewed-by: Konrad Dybcio <konrad.dybcio@oss.qualcomm.com> Reviewed-by: Vikash Garodia <vikash.garodia@oss.qualcomm.com> Signed-off-by: Dmitry Baryshkov <dmitry.baryshkov@oss.qualcomm.com> Reviewed-by: Dikshita Agarwal <dikshita.agarwal@oss.qualcomm.com>
…quency formula The driver was using a fixed default FPS value when calculating the VPU frequency. This caused wrong frequency requests for high-frame-rate streams, for example 4K at 240 FPS. Because of this, the hardware was running at a lower frequency than needed. Add the FPS measurement based on the decoder input buffer arrival rate. The measured FPS is stored per instance and used in frequency calculation instead of the fixed default FPS. The value is clamped so that it does not exceed platform limits. Add a VPP firmware overhead when running in STAGE_2. Link: https://lore.kernel.org/all/20260401-update_fps_calculation-v6-1-f44f8154ca39@oss.qualcomm.com/ Reviewed-by: Vikash Garodia <vikash.garodia@oss.qualcomm.com> Signed-off-by: Vishnu Reddy <busanna.reddy@oss.qualcomm.com>
Wire up sm8550_init_cb_devs() in iris_platform_gen2.c to register the two child context-bank devices (iris_non_pixel and iris_pixel) using iris_create_cb_dev(), and hook it into the sm8550_data platform data via the .init_cb_devs callback. Signed-off-by: Vishnu Reddy <busanna.reddy@oss.qualcomm.com> Signed-off-by: Vikash Garodia <vikash.garodia@oss.qualcomm.com> Signed-off-by: Gourav Kumar <gouravk@qti.qualcomm.com>
Different stream IDs from VPU would be associated to one of these CB. Multiple CBs are needed to increase the IOVA for the video usecases like higher concurrent sessions. Co-developed-by: Vishnu Reddy <busanna.reddy@oss.qualcomm.com> Signed-off-by: Vishnu Reddy <busanna.reddy@oss.qualcomm.com> Signed-off-by: Vikash Garodia <vikash.garodia@oss.qualcomm.com>
Depending on the buffer type (input, output and internal), associated context bank is selected, if available. Fallback to parent device for backward compatibility. Co-developed-by: Vishnu Reddy <busanna.reddy@oss.qualcomm.com> Signed-off-by: Vishnu Reddy <busanna.reddy@oss.qualcomm.com> Signed-off-by: Vikash Garodia <vikash.garodia@oss.qualcomm.com>
…ion IDs Add a dt-bindings header include/dt-bindings/media/qcom,sm8550-iris.h defining IRIS_NON_PIXEL_VCODEC (0) and IRIS_PIXEL (1) function IDs used to identify the non-pixel and pixel context bank SMMU stream mappings via iommu-map. Signed-off-by: Vishnu Reddy <busanna.reddy@oss.qualcomm.com> Signed-off-by: Vikash Garodia <vikash.garodia@oss.qualcomm.com> Signed-off-by: Gourav Kumar <gouravk@qti.qualcomm.com>
Currently the driver switches the vcodec GDSC to hardware (HW) mode before firmware load and boot sequence. GDSC can be powered off, keeping in hw mode, thereby the vcodec registers programmed in TrustZone (TZ) carry default (reset) values. Move the transition to HW mode after firmware load and boot sequence. The bug was exposed with driver configuring different stream ids to different devices via iommu-map. With registers carrying reset values, VPU would not generate desired stream-id, thereby leading to SMMU fault. For vpu4, when GDSC is switched to HW mode, there is a need to perform the reset operation. Without reset, there are occassional issues of register corruption observed. Hence the vpu GDSC switch also involves the reset. Link: https://lore.kernel.org/linux-media/20260313-kaanapali-iris-v3-2-9c0d1a67af4b@oss.qualcomm.com/ Fixes: dde659d ("media: iris: Introduce vpu ops for vpu4 with necessary hooks") Co-developed-by: Vishnu Reddy <busanna.reddy@oss.qualcomm.com> Signed-off-by: Vishnu Reddy <busanna.reddy@oss.qualcomm.com> Signed-off-by: Vikash Garodia <vikash.garodia@oss.qualcomm.com> Reviewed-by: Dikshita Agarwal <dikshita.agarwal@oss.qualcomm.com> Reviewed-by: Dmitry Baryshkov <dmitry.baryshkov@oss.qualcomm.com>
…struct The PAS ID, the authentication service ID, used by the Iris is a constant and it is not expected to change anytime. Drop it from the platform data and use the constant instead. Link: https://lore.kernel.org/linux-media/20260329-iris-platform-data-v11-1-eea672b03a95@oss.qualcomm.com/ Reviewed-by: Dikshita Agarwal <dikshita.agarwal@oss.qualcomm.com> Reviewed-by: Konrad Dybcio <konrad.dybcio@oss.qualcomm.com> Reviewed-by: Vikash Garodia <vikash.garodia@oss.qualcomm.com> Signed-off-by: Dmitry Baryshkov <dmitry.baryshkov@oss.qualcomm.com>
The set_preset_registers is (currently) common to all supported devices. Extract it to a iris_vpu_common.c and call it directly from iris_vpu_power_on(). Later, if any of the devices requires special handling, it can be sorted out separately. Link: https://lore.kernel.org/linux-media/20260329-iris-platform-data-v11-2-eea672b03a95@oss.qualcomm.com/ Reviewed-by: Dikshita Agarwal <dikshita.agarwal@oss.qualcomm.com> Reviewed-by: Vikash Garodia <vikash.garodia@oss.qualcomm.com> Signed-off-by: Dmitry Baryshkov <dmitry.baryshkov@oss.qualcomm.com>
…pecific code To note that iris_set_num_comv() is gen2-internal, rename it to iris_hfi_gen2_set_num_comv() and then stop using hfi_ops indirection to set session property (like other functions in this file do). Link: https://lore.kernel.org/linux-media/20260329-iris-platform-data-v11-3-eea672b03a95@oss.qualcomm.com/ Reviewed-by: Dikshita Agarwal <dikshita.agarwal@oss.qualcomm.com> Reviewed-by: Konrad Dybcio <konrad.dybcio@oss.qualcomm.com> Reviewed-by: Vikash Garodia <vikash.garodia@oss.qualcomm.com> Signed-off-by: Dmitry Baryshkov <dmitry.baryshkov@oss.qualcomm.com>
Calling HFI instance-specific ops should not require double indirection through the core ops. Split instance-specific ops to a separate struct, keep a pointer to it in struct iris_inst and set it directly in the get_instance function. Link: https://lore.kernel.org/linux-media/20260329-iris-platform-data-v11-4-eea672b03a95@oss.qualcomm.com/ Reviewed-by: Dikshita Agarwal <dikshita.agarwal@oss.qualcomm.com> Reviewed-by: Vikash Garodia <vikash.garodia@oss.qualcomm.com> Signed-off-by: Dmitry Baryshkov <dmitry.baryshkov@oss.qualcomm.com>
There is little point in having two different structures for HFI-related core ops. Merge both of them into the new iris_hfi_ops structure. Link: https://lore.kernel.org/linux-media/20260329-iris-platform-data-v11-5-eea672b03a95@oss.qualcomm.com/ Reviewed-by: Dikshita Agarwal <dikshita.agarwal@oss.qualcomm.com> Reviewed-by: Vikash Garodia <vikash.garodia@oss.qualcomm.com> Signed-off-by: Dmitry Baryshkov <dmitry.baryshkov@oss.qualcomm.com>
The get_instance() is a callback tightly connected to the HFI implementation. Move it into the new iris_hfi_sys_ops structure, merging all core callbacks into a single vtable. Link: https://lore.kernel.org/linux-media/20260329-iris-platform-data-v11-6-eea672b03a95@oss.qualcomm.com/ Reviewed-by: Dikshita Agarwal <dikshita.agarwal@oss.qualcomm.com> Reviewed-by: Vikash Garodia <vikash.garodia@oss.qualcomm.com> Signed-off-by: Dmitry Baryshkov <dmitry.baryshkov@oss.qualcomm.com>
…rm data The HW response time is a constant between platforms. Remove it from the iris_platform_data structure and use it directly. Link: https://lore.kernel.org/linux-media/20260329-iris-platform-data-v11-7-eea672b03a95@oss.qualcomm.com/ Suggested-by: Vikash Garodia <vikash.garodia@oss.qualcomm.com> Reviewed-by: Vikash Garodia <vikash.garodia@oss.qualcomm.com> Reviewed-by: Dikshita Agarwal <dikshita.agarwal@oss.qualcomm.com> Signed-off-by: Dmitry Baryshkov <dmitry.baryshkov@oss.qualcomm.com>
Having firmware-related fields in platform data results in the tying platform data to the HFI firmware data rather than the actual hardware. For example, SM8450 uses Gen2 firmware, so currently its platform data should be placed next to the other gen2 platforms, although it has the VPU2.0 core, similar to the one found on SM8250 and SC7280 and so the hardware-specific platform data is also close to those devices. Split firmware data to a separate struct, separating hardware-related data from the firmware interfaces. Link: https://lore.kernel.org/linux-media/20260329-iris-platform-data-v11-8-eea672b03a95@oss.qualcomm.com/ Reviewed-by: Dikshita Agarwal <dikshita.agarwal@oss.qualcomm.com> Signed-off-by: Dmitry Baryshkov <dmitry.baryshkov@oss.qualcomm.com>
Finalize the logical separation of the software and hardware interface descriptions by moving hardware properties to the files specific to the particular VPU version. Link: https://lore.kernel.org/linux-media/20260329-iris-platform-data-v11-9-eea672b03a95@oss.qualcomm.com/ Reviewed-by: Dikshita Agarwal <dikshita.agarwal@oss.qualcomm.com> Signed-off-by: Dmitry Baryshkov <dmitry.baryshkov@oss.qualcomm.com>
The linux-firmware is providing the vpuNN_pM.mbn firmware for SM8250 since August of 2024. Stop using the legacy firmware name (vpu-1.0/venus.mbn) and switch to the standard firmware name schema (vpu/vpu20_p4.mbn). Link: https://lore.kernel.org/linux-media/20260329-iris-platform-data-v11-10-eea672b03a95@oss.qualcomm.com/ Reviewed-by: Vikash Garodia <vikash.garodia@oss.qualcomm.com> Reviewed-by: Dikshita Agarwal <dikshita.agarwal@oss.qualcomm.com> Signed-off-by: Dmitry Baryshkov <dmitry.baryshkov@oss.qualcomm.com>
In preparation to adding support for several firmware revisions to be used for a platform, extract the firmware description data. It incorporates firmware name, HFI ops and buffer requirements of the particular firmware build. Link: https://lore.kernel.org/linux-media/20260329-iris-platform-data-v11-11-eea672b03a95@oss.qualcomm.com/ Reviewed-by: Dikshita Agarwal <dikshita.agarwal@oss.qualcomm.com> Signed-off-by: Dmitry Baryshkov <dmitry.baryshkov@oss.qualcomm.com>
UBWC configuration is not applicable to all SoCs. Add a check to avoid configuring UBWC during sys init on unsupported platforms. Reviewed-by: Konrad Dybcio <konrad.dybcio@oss.qualcomm.com> Signed-off-by: Dikshita Agarwal <dikshita.agarwal@oss.qualcomm.com>
The raw formats supported by Iris were previously advertised unconditionally, assuming UBWC support on all platforms. However, some platforms do not support UBWC which results in incorrect format capability exposure. Use the UBWC configuration provided by the platform to dynamically filter raw formats at runtime. If UBWC is not supported, UBWC-based formats are omitted from the advertised capability list, while linear formats remain available. Signed-off-by: Dikshita Agarwal <dikshita.agarwal@oss.qualcomm.com>
The set_preset_registers sequence is currently shared across all supported devices. Starting with Qualcomm QCM2290 (AR50LT), the register programming would differ. Move set_preset_register into a vpu_op to allow per-device customization. This change prepares the driver for upcoming hardware variants. No functional change so far for existing devices. Reviewed-by: Dmitry Baryshkov <dmitry.baryshkov@oss.qualcomm.com> Signed-off-by: Dikshita Agarwal <dikshita.agarwal@oss.qualcomm.com>
The interrupt_init sequence is currently shared across all supported devices. Starting with Qualcomm QCM2290 (AR50LT), the register programming would differ. Move interrupt_init into a vpu_op to allow per-device customization. This change prepares the driver for upcoming hardware variants. No functional change so far for existing devices. Reviewed-by: Dmitry Baryshkov <dmitry.baryshkov@oss.qualcomm.com> Signed-off-by: Dikshita Agarwal <dikshita.agarwal@oss.qualcomm.com>
On AR50LT platforms AbsolutelyPerfectRouting (ARP) needs to be disabled so firmware can configure the ARP internal buffer as non-secure for encoder usage. In preparation of adding support for AR50LT platforms, add an optional disable_arp callback to the VPU ops and invoke it from core init and resume paths. No functional change for existing platforms. Reviewed-by: Dmitry Baryshkov <dmitry.baryshkov@oss.qualcomm.com> Signed-off-by: Dikshita Agarwal <dikshita.agarwal@oss.qualcomm.com>
For AR50LT core, the value of WRAPPER_INTR_STATUS_A2HWD_BMASK differs from the currently supported VPUs. In preparation for adding AR50LT support in subsequent patches, introduce a platform data field, wd_intr_mask, to capture the watchdog interrupt bitmask per platform. Signed-off-by: Dikshita Agarwal <dikshita.agarwal@oss.qualcomm.com>
AR50LT require explicit instantaneous bandwidth (IB) voting in addition to average bandwidth (AB) when configuring interconnect QoS. This requirement is due to QSB (Qualcomm System Bus) 128b to QNS ( Qualcomm Network Switch) 256b conversion at video noc in AR50LT which is not needed for other IRIS cores. In preparation of adding support for AR50LT core, introduce platform-configurable IB multiplier and enable IB voting for all SoCs. Existing platforms default to IB == AB, while AR50LT requires 2x peak bandwidth. Signed-off-by: Dikshita Agarwal <dikshita.agarwal@oss.qualcomm.com>
Add power sequence for ar5lt core. Add register handling for ar50lt by hooking up vpu op with ar50lt specific implemtation or resue from earlier generation wherever feasible. Signed-off-by: Dikshita Agarwal <dikshita.agarwal@oss.qualcomm.com>
Introduces AR50LT buffer size calculation for both encoder and decoder. Reuse the buffer size calculation which are common, while adding the AR50LT specific ones separately. Signed-off-by: Dikshita Agarwal <dikshita.agarwal@oss.qualcomm.com>
Add support for the qcm2290 SoC by introducing a new compatible string for the AR50LT core and the corresponding platform data. This change: - Adds qcm2290 as a supported compatible for the AR50LT core. - Introduces AR50LT-specific platform data describing VPU configuration. - Adds a qcm2290-specific header describing firmware capabilities. Signed-off-by: Dikshita Agarwal <dikshita.agarwal@oss.qualcomm.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR includes the changes which are missing on this branch from video tech kernel-topics branch and the changes which are needed to enable video on Shikra