{"id":417,"date":"2022-05-12T17:13:35","date_gmt":"2022-05-12T15:13:35","guid":{"rendered":"https:\/\/threedots.ovh\/blog\/?p=417"},"modified":"2022-05-15T19:27:00","modified_gmt":"2022-05-15T17:27:00","slug":"nvidias-open-source-drivers","status":"publish","type":"post","link":"https:\/\/threedots.ovh\/blog\/2022\/05\/nvidias-open-source-drivers\/","title":{"rendered":"NVIDIA&#8217;s open-source drivers"},"content":{"rendered":"\n<p>NVIDIA has multiple different open-source kernel drivers for their GPU and display hardware, used in different scenarios &#8211; and working on different sets of NVIDIA devices.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">nvgpu driver<\/h2>\n\n\n\n<p>The <code>nvgpu<\/code> driver is used mainly\/almost exclusively on NVIDIA Tegra SoCs. It&#8217;s a GPU &#8211; as in accelerator block &#8211; only driver that doesn&#8217;t handle display management &#8211; as that&#8217;s handled by separate display hardware.<\/p>\n\n\n\n<p>This driver is available at <a href=\"https:\/\/nv-tegra.nvidia.com\/r\/gitweb?p=linux-nvgpu.git;a=summary\">https:\/\/nv-tegra.nvidia.com\/r\/gitweb?p=linux-nvgpu.git;a=summary<\/a> under MIT license, except the Linux-specific components, which are licensed as GPLv2.<\/p>\n\n\n\n<p>For a Tegra integrated GPUs, <code>nvgpu<\/code> isn&#8217;t a standalone driver: it relies on the <code>nvmap<\/code> and <code>host1x<\/code> drivers for memory mapping and communication respectively.<\/p>\n\n\n\n<p>For dedicated GPUs, unified memory isn&#8217;t supported with this driver. NVIDIA also doesn&#8217;t redistribute a firmware set for this driver for dedicated GPUs.<\/p>\n\n\n\n<p><code>nvgpu<\/code> supports multiple operating systems. It runs on Linux, QNX and Nintendo Horizon. This drivers supports Functional Safety (FUSA) requirements for automotive use.<\/p>\n\n\n\n<p>This driver doesn&#8217;t require the usage of the GPU System Processor present on Turing and later, and supports earlier GPUs.<\/p>\n\n\n\n<p>Supported iGPUs by this driver: GM20B, GP10B, GV11B, GA10B<\/p>\n\n\n\n<p>Supported dGPUs by this driver: GP106, GV100, TU104, GA100<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Firmware set for Xavier (nvgpu)<\/h3>\n\n\n\n<pre class=\"wp-block-code\"><code>-rwxr-xr-x  1 root root 102K avril  6 21:08 NETA_img.bin\n-rwxr-xr-x  1 root root 102K avril  6 21:08 NETB_img.bin\n-rwxr-xr-x  1 root root 102K avril  6 21:08 NETC_img.bin\n-rw-r--r--  1 root root  97K avril  6 21:08 NETD_img.bin\n-rw-r--r--  1 root root  25K avril  6 21:08 acr_ucode_dbg.bin\n-rw-r--r--  1 root root  25K avril  6 21:08 acr_ucode_prod.bin\n-rw-r--r--  1 root root  528 avril  6 21:08 fecs.bin\n-rw-r--r--  1 root root  108 avril  6 21:08 fecs_sig.bin\n-rw-r--r--  1 root root  528 avril  6 21:08 gpccs.bin\n-rw-r--r--  1 root root  108 avril  6 21:08 gpccs_sig.bin\n-rw-r--r--  1 root root  95K avril  6 21:08 gpmu_ucode.bin\n-rw-r--r--  1 root root  656 avril  6 21:08 gpmu_ucode_desc.bin\n-rw-r--r--  1 root root  95K avril  6 21:08 gpmu_ucode_image.bin\n-rw-r--r--  1 root root 1,3K avril  6 21:08 pmu_bl.bin\n-rw-r--r--  1 root root  108 avril  6 21:08 pmu_sig.bin<\/code><\/pre>\n\n\n\n<h2 class=\"wp-block-heading\">Tegra DC driver<\/h2>\n\n\n\n<p><code>tegradc<\/code> is a driver for the display controller used on NVIDIA Tegra hardware prior to Orin (t234). It&#8217;s fully independent from accelerator hardware.<\/p>\n\n\n\n<p>The mainline kernel uses a DRM <code>tegra<\/code> driver for the same purpose &#8211; which also has support for 3D acceleration on hardware that pre-dates Tegra K1. <\/p>\n\n\n\n<p>The K1 switched to regular NVIDIA GPU architectures instead of using GeForce ULP.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">nvidia driver<\/h2>\n\n\n\n<p>Those drivers below are derived from the same root codebase and share the same module names. (<code>nvidia<\/code> and <code>nvidia-*<\/code>) As such, they cannot be loaded at the same time as one another. You&#8217;ll have to pick <em>one<\/em> of those on a given machine.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Proprietary<\/h3>\n\n\n\n<p>The NVIDIA proprietary driver has obfuscated symbols, to (try to) make reverse engineering harder than it could be. Non-GSP firmware is directly included in the binary instead of being separate, unlike <code>nvgpu<\/code> which has it stored it in <code>\/lib\/firmware<\/code>. <\/p>\n\n\n\n<p>The proprietary component code sizes for this kernel driver:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>1,5M nvidia-modeset\/nv-modeset-kernel.o_binary\n44M nvidia\/nv-kernel.o_binary<\/code><\/pre>\n\n\n\n<p>The ones on the <em>Open GPU Kernel Modules <\/em>for comparison, when precompiled by NVIDIA for shorter installation times:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>1,8M nvidia-modeset\/nv-modeset-kernel.o_binary\n6,3M nvidia\/nv-kernel.o_binary<\/code><\/pre>\n\n\n\n<h4 class=\"wp-block-heading\">GPU System Processor<\/h4>\n\n\n\n<p>This driver does support using the GPU System Processor, which is available on Turing GPUs (GeForce GTX 16xx and RTX) onwards.<\/p>\n\n\n\n<p>The GSP firmware is totally optional in this driver. It is not required to be used in any way. <\/p>\n\n\n\n<p>When the GSP driver is used, a significant chunk of the driver is offloaded, which makes open-sourcing that code not necessary in order to not have proprietary code running in the host OS kernel. <\/p>\n\n\n\n<p>An extensive RPC interface is used for communication across the two worlds, not very much unlike what Apple does for the display controller on the A14 onwards.<\/p>\n\n\n\n<p>The GSP firmware weights 39MB on the latest <code>515.43.04<\/code> driver release.<\/p>\n\n\n\n<p>It&#8217;s a RISC-V 64-bit ELF binary.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>firmware\/gsp.bin: ELF 64-bit LSB executable, UCB RISC-V, version 1 (SYSV), statically linked, stripped<\/code><\/pre>\n\n\n\n<p>Sections on the <code>gsp.bin<\/code> firmware file:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>Section Headers:\n  &#91;Nr] Name              Type            Address          Off    Size   ES Flg Lk Inf Al\n  &#91; 0]                   NULL            0000000000000000 000000 000000 00      0   0  0\n  &#91; 1] .gnext_data       PROGBITS        0000000004001000 001000 0021a8 00  WA  0   0  8\n  &#91; 2] .ga10x_data       PROGBITS        0000000004004000 004000 001090 00  WA  0   0  8\n  &#91; 3] .ga100_data       PROGBITS        0000000004006000 006000 0010f0 00  WA  0   0  8\n  &#91; 4] .tu10x_data       PROGBITS        0000000004008000 008000 0010f0 00  WA  0   0  8\n  &#91; 5] .gnext_text       PROGBITS        000000000400a000 00a000 001830 00  AX  0   0  4\n  &#91; 6] .ga10x_text       PROGBITS        000000000400c000 00c000 00107c 00  AX  0   0  4\n  &#91; 7] .ga100_text       PROGBITS        000000000400e000 00e000 001104 00  AX  0   0  4\n  &#91; 8] .tu10x_text       PROGBITS        0000000004010000 010000 001104 00  AX  0   0  4\n  &#91; 9] .bootstrap        PROGBITS        0000000004012000 012000 0000c4 00  AX  0   0  4\n  &#91;10] .gnext_resident_text PROGBITS     0000000004013000 013000 002000 00 WAX  0   0  4\n  &#91;11] .gnext_resident_data PROGBITS     0000000004015000 015000 001000 00  WA  0   0 32\n  &#91;12] .ga10x_resident_text PROGBITS     0000000004016000 016000 002000 00 WAX  0   0  4\n  &#91;13] .ga10x_resident_data PROGBITS     0000000004018000 018000 001000 00  WA  0   0 32\n  &#91;14] .ga100_resident_text PROGBITS     0000000004019000 019000 002000 00 WAX  0   0  4\n  &#91;15] .ga100_resident_data PROGBITS     000000000401b000 01b000 001000 00  WA  0   0 32\n  &#91;16] .tu10x_resident_text PROGBITS     000000000401c000 01c000 002000 00 WAX  0   0  4\n  &#91;17] .tu10x_resident_data PROGBITS     000000000401e000 01e000 001000 00  WA  0   0 32\n  &#91;18] .acl              PROGBITS        000000000401f000 01f000 000160 00  WA  0   0  1\n  &#91;19] .manifest         PROGBITS        000000000401f160 01f160 000ea0 00  WA  0   0  1\n  &#91;20] .section_task_init_stack_instance PROGBITS 000000000662a000 021000 002000 00  WA  0   0  1\n  &#91;21] .section_task_rm_stack_instance NOBITS 000000000662d000 967000 010000 00  WA  0   0  1\n  &#91;22] .header           PROGBITS        0000000004020000 020000 000010 00  WA  0   0  1\n  &#91;23] .memory_region_init_arguments PROGBITS 0000000004020010 263c010 000060 00  WA  0   0  1\n  &#91;24] .section_task_rm_elf_rodata_instance PROGBITS 0000000004021000 97e000 11d000 00   A  0   0  8\n  &#91;25] .section_task_rm_elf_data_instance PROGBITS 000000000413e000 a9f000 1b9d000 00  WA  0   0  8\n  &#91;26] .section_task_rm_elf_paged_data_instance PROGBITS 0000000005cdb000 263c070 000000 00   W  0   0  1\n  &#91;27] .section_task_rm_elf_text_instance PROGBITS 0000000005cdb000 023000 944000 00  AX  0   0  4\n  &#91;28] .section_task_rm_elf_paged_text_instance PROGBITS 000000000661f000 263c070 000000 00   W  0   0  1\n  &#91;29] .section_task_init_elf_rodata_instance PROGBITS 000000000661f000 97d000 001000 00   A  0   0  8\n  &#91;30] .section_task_init_elf_data_instance PROGBITS 0000000006620000 967000 001000 00  WA  0   0  8\n  &#91;31] .fwsignature_ga100 PROGBITS       0000000006621000 968000 001000 00  WA  0   0  8\n  &#91;32] .fwsignature_ga10x PROGBITS       0000000006622000 969000 001000 00  WA  0   0  8\n  &#91;33] .fwsignature_tu10x PROGBITS       0000000006623000 96a000 001000 00  WA  0   0  8\n  &#91;34] .fwsignature_tu11x PROGBITS       0000000006624000 96b000 001000 00  WA  0   0  8\n  &#91;35] .section_task_init_elf_paged_data_instance PROGBITS 0000000006625000 a9b000 001000 00  WA  0   0  1\n  &#91;36] .section_task_init_elf_text_instance PROGBITS 0000000006626000 a9c000 003000 00  AX  0   0  4\n  &#91;37] .section_task_init_elf_paged_text_instance PROGBITS 0000000006629000 97c000 001000 00  WA  0   0  1\n  &#91;38] .section_task_rm_memset_buffer_instance NOBITS 000000000662c000 023000 001000 00  WA  0   0  1\n  &#91;39] .section_log_rm_buffer_instance NOBITS 000000000663d000 000000 040000 00  WA  0   0  1\n  &#91;40] .section_task_rm_args_buffer_instance NOBITS 000000000667d000 000000 001000 00  WA  0   0  1\n  &#91;41] .section_task_init_dmem_64kb_instance PROGBITS 000000000667e000 96c000 010000 00  WA  0   0  1\n  &#91;42] .section_log_init_buffer_instance NOBITS 000000000668e000 000000 010000 00  WA  0   0  1\n  &#91;43] .section_task_rm_message_timing_buffer_instance NOBITS 000000000669e000 97d000 001000 00  WA  0   0  1\n  &#91;44] .section_task_rm_msgq_work_areas_instance NOBITS 000000000669f000 a9c000 021000 00  WA  0   0  1\n  &#91;45] .section_task_rm_memcpy_buffer_instance NOBITS 00000000066c0000 a9c000 001000 00  WA  0   0  1\n  &#91;46] .comment          PROGBITS        0000000000000000 263c070 000011 01  MS  0   0  1\n  &#91;47] .shstrtab         STRTAB          0000000000000000 263c081 0004a6 00      0   0  1\nKey to Flags:\n  W (write), A (alloc), X (execute), M (merge), S (strings), I (info),\n  L (link order), O (extra OS processing required), G (group), T (TLS),\n  C (compressed), x (unknown), o (OS specific), E (exclude),\n  R (retain), p (processor specific)<\/code><\/pre>\n\n\n\n<p>We can see that most executable code is shared between GPU generations in this scenario.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Open GPU Kernel Modules<\/h3>\n\n\n\n<p>This driver is the one above, but with <em>only <\/em>supporting the GPU System Processor offloaded code path and available under an open-source license (dual GPLv2 + MIT).<\/p>\n\n\n\n<p>As it relies on the GSP, this driver only supports Turing and later.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">opensrc-disp<\/h3>\n\n\n\n<p><code>opensrc-disp<\/code> is derived from the mainline NVIDIA closed-source driver, like the one above. It replaces <code>tegradc<\/code> on Orin (t234) and later. It does not drive the GPU accelerator, which is still driven by the <code>nvgpu<\/code> driver on Orin.<\/p>\n\n\n\n<p>Currently, this driver only supports T234D and as such, when using a dedicated GPU on Orin, the onboard display controller is lost.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Userspace compatibility<\/h2>\n\n\n\n<p>No ABI compatibility is guaranteed between kernel and user land across <code>nvidia<\/code> GPU driver versions. <code>nvgpu<\/code> has a significantly higher ABI stability level.<\/p>\n\n\n\n<p><code>nvgpu<\/code> and <code>nvidia<\/code> do not share ABIs. As such, different sets of user-mode drivers are needed to interface with them. <\/p>\n\n\n\n<p>In both the <code>nvgpu<\/code> and <code>nvidia<\/code> scenarios, the user-space GPU driver components provided by NVIDIA are proprietary.<\/p>\n\n\n\n<p><em>edit (May 15): all of the <code>nvgpu<\/code> Linux-specific components are licensed as GPLv2.<\/em><\/p>\n","protected":false},"excerpt":{"rendered":"<p>NVIDIA has multiple different open-source kernel drivers for their GPU and display hardware, used in different scenarios &#8211; and working on different sets of NVIDIA devices. nvgpu driver The nvgpu driver is used mainly\/almost exclusively on NVIDIA Tegra SoCs. It&#8217;s a GPU &#8211; as in accelerator block &#8211; only driver that doesn&#8217;t handle display management&hellip;&nbsp;<a href=\"https:\/\/threedots.ovh\/blog\/2022\/05\/nvidias-open-source-drivers\/\" rel=\"bookmark\">Read More &raquo;<span class=\"screen-reader-text\">NVIDIA&#8217;s open-source drivers<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"neve_meta_sidebar":"","neve_meta_container":"","neve_meta_enable_content_width":"","neve_meta_content_width":0,"neve_meta_title_alignment":"","neve_meta_author_avatar":"","neve_post_elements_order":"","neve_meta_disable_header":"","neve_meta_disable_footer":"","neve_meta_disable_title":"","footnotes":""},"categories":[1],"tags":[],"class_list":["post-417","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"_links":{"self":[{"href":"https:\/\/threedots.ovh\/blog\/wp-json\/wp\/v2\/posts\/417","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/threedots.ovh\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/threedots.ovh\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/threedots.ovh\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/threedots.ovh\/blog\/wp-json\/wp\/v2\/comments?post=417"}],"version-history":[{"count":6,"href":"https:\/\/threedots.ovh\/blog\/wp-json\/wp\/v2\/posts\/417\/revisions"}],"predecessor-version":[{"id":440,"href":"https:\/\/threedots.ovh\/blog\/wp-json\/wp\/v2\/posts\/417\/revisions\/440"}],"wp:attachment":[{"href":"https:\/\/threedots.ovh\/blog\/wp-json\/wp\/v2\/media?parent=417"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/threedots.ovh\/blog\/wp-json\/wp\/v2\/categories?post=417"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/threedots.ovh\/blog\/wp-json\/wp\/v2\/tags?post=417"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}