[Feature Request] Build Environment Build Customization

[rrrix]

Firstly, I've been watching relenv for a while - I think it's rad. Great work 😎

I would like to add support for customizing the build environment (e.g. what's created with relenv build). Specifically:

• Custom [Cc]onfigure flags for CPython, OpenSSL, SQLite, etc.
• Custom CFLAGS / LDFLAGS / CXXFLAGS
• Other Custom environment variables (e.g. MACOSX_DEPLOYMENT_TARGET)
• Make flags
• Customization of source tarballs (Download URL's, Versions, Checksums)

Although my primary focus is on macOS (as I support my Company's fleet of macOS devices, including its use of Salt), I think customization would be valuable for all platforms.

As a good example, take issue #114, where the default /usr/bin/python3 executable is hard-coded in relenv, and is actually immutable on macOS (due to System Integrity Protection), but can "point" to different Xcode environments (via xcode-select).

If I wanted to use zlib or libzmq from Homebrew (instead of building it myself), currently relenv doesn't allow me to do that.

I want to be able use a different Python version (e.g. a pre-release version, 3.9.x, or any other version not specifically hard-coded into relenv.

[dwoz]

I'm not opposed to any of this but we do need to think about everything carefully. There is also a fair amount to un-pack here and I think it will potentially warrant multiple issues. For now, I'll try to provide my thoughts thus far.

I would like to add support for customizing the build environment (e.g. what's created with relenv build). Specifically:

• Custom [Cc]onfigure flags for CPython, OpenSSL, SQLite, etc.

Seems reasonable but I'm not sure the best way to add this. Thoughts that come to mind are; command line arguments, environment variables, some kind of config file, being able to extend/modify the build with python.

• Custom CFLAGS / LDFLAGS / CXXFLAGS

I think this would work today on windows because we consume the existing environment, there could be a flag to do that on the other platforms.

• Other Custom environment variables (e.g. MACOSX_DEPLOYMENT_TARGET)

I see this in a similar light as the configure flags.. Could be handled in any of the ways listed above.

• Make flags

What are some examples of this?

• Customization of source tarballs (Download URL's, Versions, Checksums)

Not sure what you are after here. Some examples might be helpful. Though this brings up a point I want to make about all of this. If we start customizing builds we're going to need a way to differentiate those customized builds from the 'official' builds. I see this as being similar to how pyenv has multiple variants of python you can install. Would relenv be on the hook for building and hosing these builds? Would there be a mechanism for others to host them and how would relenv find them?

Although my primary focus is on macOS (as I support my Company's fleet of macOS devices, including its use of Salt), I think customization would be valuable for all platforms.

As a good example, take issue #114, where the default /usr/bin/python3 executable is hard-coded in relenv, and is actually immutable on macOS (due to System Integrity Protection), but can "point" to different Xcode environments (via xcode-select).

While addressing #114 I tried pulling python from the path. This caused problems in the CI/CD pipelines for salt because we're using virtual environments and in a virtualenv the python in the path points back to the relenv interpreter. This was a no-go so I backed that logic out. Possibly a better way to handle this is an environment variable to tell relenv where to look for the system python?

If I wanted to use zlib or libzmq from Homebrew (instead of building it myself), currently relenv doesn't allow me to do that.

This is interesting, I see where there could be value and it has actually come up before. The one thing that worries my is it breaks some expectations relenv currently has about portability. This may not be an issue on OSX but it is on linux.

I want to be able use a different Python version (e.g. a pre-release version, 3.9.x, or any other version not specifically hard-coded into relenv.

This has also come up before, It wouldn't be hard to add a command line flag for this to build, but then we need to figure out how to support relenv create. If there are specific versions you have in mind we can discuss adding them as fully supported in relenv. As of now, I'm only looking at supporting the latest minor releases of the most current python major versions.

For all of this stuff, and specifically around flags for Sqlite and the likes, there will likely be things that make sense to add to the official builds.

Finally, Thank you very much for the feedback. I'd be very happy to see relenv get adopted outside of the usage in the Salt project.