Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

blender: add option for OptiX #106888

Merged
merged 4 commits into from Jan 8, 2021
Merged

blender: add option for OptiX #106888

merged 4 commits into from Jan 8, 2021

Conversation

NomisIV
Copy link
Contributor

@NomisIV NomisIV commented Dec 14, 2020

Motivation for this change

Added support for OptiX in blender. I added it as an option, just like cudaSupport, so that it wouldn't be mandatory.

I have successfully built this package on my own machine (running NixOS). I don't think the changes are big enough to cause any major problems.

Something worth noting is that this is done in the same way as Arch linux has implemented it: link

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
    (I could successfully build it with a recent clone of nixpkgs)
  • Tested execution of all binary files (usually in ./result/bin/)
    (Blender started, and the OptiX render option is both visible and working)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.
    (Commit message follows the guide lines, I think)

@InternetUnexplorer
Copy link
Contributor

Thanks for doing this. I'm wondering if it would be a good idea to remove the optixSupport option entirely and just have it controlled by cudaSupport, as

  • OptiX is now supported on most recent NVIDIA GPUs (instead of just Turing and later),
  • It would be one less option to enable, and
  • It would prevent people from enabling OptiX support without enabling CUDA support.

I don't feel very strongly about it, but if it seems like a good idea to keep the option, then maybe we should consider having it follow cudaSupport by default (making it opt-out instead of opt-in) and/or ensuring that it can't be enabled without enabling cudaSupport as well.

@NomisIV
Copy link
Contributor Author

NomisIV commented Dec 14, 2020

Good point. My reasoning at the time was that you might want to enable CUDA support without enabling OptiX, however, as you mentioned those cases would be quite rare. I think integrating the OptiX into the CUDA setting is the way to go, since it removes a useless setting (optixSupport), and anyone wanting to use OptiX would only have to update nixpkgs and not fiddle with another override setting.

I'm guessing I delete the pull request and open a new one with the changes, or can I simply add another commit to the existing pull request?
Also, do you (@InternetUnexplorer) have the authority to merge this when you're satisfied, or does someone "higher ranked" have to do that?
This is my first contribution, hence my lack of experience.

@InternetUnexplorer
Copy link
Contributor

I'm guessing I delete the pull request and open a new one with the changes, or can I simply add another commit to the existing pull request?
Also, do you have the authority to merge this when you're satisfied, or does someone "higher ranked" have to do that?

You should be able to just push another commit to NomisIV:master and it will show up here.
As for me, I'm totally new to contributing as well, but the maintainers are pretty responsive so it shouldn't take too long for them to come by and take a look at it. You might want to @mention or request a review from the package maintainers, but I don't know how necessary that is.
Worst case, if a week or so goes by with no activity there's a thread on the NixOS Discourse where you can post PRs that haven't received any attention, but someone should hopefully come by before then.

Copy link
Member

@veprbl veprbl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like the headers pulled when optixSupport = true are not free. I think we should update the blender.meta.license field to indicate that.
Perhaps something like

meta = {
  # skip
  license = with licenses; [ gpl2Plus ] ++ optional optixSupport unfree;
}

@NomisIV
Copy link
Contributor Author

NomisIV commented Dec 15, 2020

Considering the license, it would make more sense (than before) to have optixSupport as a separate setting, since the cudaSupport doesn't affect the license. Not sure if I have convinced myself though. Any opinions? Personally, I still vote to merge it with cudaSupport.

I'm working on the changes, and they should be online soon

EDIT: Another idea could be to make optixSupport follow cudaSupport by default (as @InternetUnexplorer proposed), that way it would be possible to have cudaSupport enabled while still having a free license. But again, I don't think that that user base would be enormous.

@veprbl
Copy link
Member

veprbl commented Dec 15, 2020

nix-repl> (import <nixpkgs> {}).cudatoolkit 
error: Package ‘cudatoolkit-10.2.89’ in /Users/veprbl/nixpkgs/pkgs/development/compilers/cudatoolkit/common.nix:214 has an unfree license (‘unfree’), refusing to evaluate.

Copy link
Member

@veprbl veprbl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@InternetUnexplorer
Copy link
Contributor

InternetUnexplorer commented Dec 15, 2020

The license situation is definitely confusing to me; Arch has the package license as GPL even though it includes OptiX, the official Blender binaries are built with OptiX support, and I recall this tweet which says:

Optix is now compatible with GPL, a well integrated version for Cycles is being worked on.

That said, being "compatible with" GPL doesn't mean it is GPL, and so I'm not sure what all of this means for the package license.

Edit: reading the license comment at the top of the headers, they definitely seem to be unfree, so it seems like @veprbl is right.

@InternetUnexplorer
Copy link
Contributor

nix-repl> (import <nixpkgs> {}).cudatoolkit 
error: Package ‘cudatoolkit-10.2.89’ in /Users/veprbl/nixpkgs/pkgs/development/compilers/cudatoolkit/common.nix:214 has an unfree license (‘unfree’), refusing to evaluate.

Oh, I didn't even think of that. Good point. In that case, I'm personally in favor of keeping the option and having it follow cudaSupport by default, since:

  • Anyone who has enabled cudaSupport will already have allowUnfree = true,
  • Most users with NVIDIA GPUs are able to use OptiX but they might not know about the option, so having it be on by default seems like a good idea, and
  • Anyone who doesn't want Blender itself to be unfree can always opt out.

Again, I don't feel very strongly about this, just putting my thoughts out there.

@InternetUnexplorer
Copy link
Contributor

One last thing: I mentioned this earlier, but would it be worth it to ensure that Blender cannot be built with optixSupport = true when cudaSupport = false? I'm not familiar enough with nixpkgs to know the "correct" way of handling conflicting options, if any.

I think the build will still succeed in this situation (although I'm not able to check right now), OptiX just won't work obviously.

@veprbl
Copy link
Member

veprbl commented Dec 15, 2020

IMHO less options is better because they don't need to be tested if they don't exist. Having said that, I don't know how optix is related to CUDA.

@InternetUnexplorer
Copy link
Contributor

InternetUnexplorer commented Dec 15, 2020

Having said that, I don't know how optix is related to CUDA.

I'm confused about it as well, so I looked into it a bit. Here's what I've learned so far (may be wrong!):

  • Enabling CUDA support (WITH_CYCLES_DEVICE_CUDA) requires the CUDA headers to be available, unless WITH_CUDA_DYNLOAD is enabled.
  • Enabling OptiX support (WITH_CYCLES_DEVICE_OPTIX) requires the OptiX headers to be available, but does not require the CUDA headers to be available if WITH_CUDA_DYNLOAD is enabled.
  • If WITH_CYCLES_CUDA_BINARIES is enabled and cudatoolkit is available then the CUDA kernels will be compiled at build time, including the OptiX kernels if OptiX is enabled and OPTIX_ROOT_DIR is set.
  • If WITH_CYCLES_CUDA_BINARIES is not enabled or cudatoolkit is not available then both the CUDA/OptiX headers can be built on demand if cudatoolkit and the OptiX SDK are available. The OptiX support is relatively new and I'm not sure how it works.
  • OptiX itself seems to be part of the NVIDIA driver.

So, it seems like it's pretty complicated. It doesn't seem like it's possible to enable OptiX support without the OptiX headers, but it does seem possible to build OptiX/CUDA support without the CUDA headers.

Also, while it's not strictly related to this PR, it seems like it's possible to build Blender with dynamic support with CUDA. I might make a separate PR for enabling that, since it would mean that Blender would not need to be rebuilt when someone enables config.cudaSupport.

@symphorien
Copy link
Member

Shouldn't optix go into its own file and own attribute in nixpkgs, so that is can be reused if other software needs it?

@InternetUnexplorer
Copy link
Contributor

InternetUnexplorer commented Dec 15, 2020

Apparently it's not possible due to legal reasons, at least according to this comment in the Arch PKGBUILD.

@symphorien
Copy link
Member

It's probably because the headers are not redistributable, but this is not a problem with nix if the derivation is properly marked as unfree, I think.

@prusnak prusnak linked an issue Dec 16, 2020 that may be closed by this pull request
@nixos-discourse
Copy link

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/prs-ready-for-review/3032/417

@prusnak
Copy link
Member

prusnak commented Dec 30, 2020

@NomisIV Can you please update the PR? Latest OptiX is 7.2.0 now ..

Co-authored-by: Pavol Rusnak <pavol@rusnak.io>
@NomisIV
Copy link
Contributor Author

NomisIV commented Dec 30, 2020

The arch package, which I used for reference, is not yet updated to use OptiX 7.2.0, so I can't just copy their new link and checksum. I'm away from home over New Year's, but in a couple of days I'll get back home and be able to fix it.

@NomisIV
Copy link
Contributor Author

NomisIV commented Jan 8, 2021

Regarding the update to OptiX 7.2, I tried my best but I remain defeated. I tried simply bumping the numbers in the url to the library, but that didn't work. I went as far as to make an NVidia Developer account to be able to download the right version, but all that gave me access to was a shell script with a tarball appended to the end of the file. I could upload it somewhere for someone to analyze, but there was a pretty scary looking license in the file as well, so I don't want to be the one doing that. And since Arch Linux hasn't updated to 7.2 either, I don't think it's a massive problem. How shall we proceed?

@prusnak
Copy link
Member

prusnak commented Jan 8, 2021

Let's proceed with 7.0. Thank you for trying OptiX 7.2!

@prusnak prusnak merged commit 9ff1aa8 into NixOS:master Jan 8, 2021
InternetUnexplorer added a commit to InternetUnexplorer/nixpkgs-overlay that referenced this pull request Jan 11, 2021
OptiX support was added to Nixpkgs in
NixOS/nixpkgs#106888.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

OptiX in Blender
7 participants