Replies: 31 comments 34 replies
-
|
Is it possible to request an exclusion for this? The issue is that we use the Maven resolver API for this, so this is entirely controlled by Maven's internals. I am not sure if there is an API to control this without reimplementing large chunks of Aether. Another workaround could be to use a URL or protoc on the system PATH potentially, as those download to target/protobuf-maven-plugin. Failing that, you can make a file in the root of your repository named ...which will rewrite the m2 location for just the current process. Would any of these work for you? Appreciate it is not an ideal solution. |
Beta Was this translation helpful? Give feedback.
-
Not really since it's centrally and globally managed.
I can't actually see in there.
This would then not work on the CI/CD pipeline 'cause I have a specific Maven repository folder there and everything else is read-only. Moreover, I can't change it on the fly since it's part of the GitLab runner configuration in OCP. @ascopes can you maybe reuse https://maven.apache.org/plugins/maven-dependency-plugin/usage.html#dependency:copy for the download part? |
Beta Was this translation helpful? Give feedback.
-
|
Whilst dependency:copy could be used in theory, it results in a fairly large amount of refactoring of the entire plugin since this is not going to work nicely with protoc plugins. Protoc plugins are also expected to be native executables to work with protoc, and being able to invoke executables from m2 is a fairly common accepted pattern (e.g. gRPC does this, Maven wrapper does this; I believe the old Xolstice plugin also does this although I have not checked. The os72 plugin distributed specific versions of protobuf within a jar but lacked any of the dependency resolution framework we use). With Java based plugins this potentially can cause other issues with various internal resolution logic. The other problem I can see is this would still cater to a very distinct use case which would not be possible to satisfy for everyone. For example, other orgs may disallow executables being placed outside m2. It is probably worth noting though that if your org is disallowing the download of executables in general, it is probably going to be better in the long run to use protoc installed on the system path within CI and locally, since that way you are using an internally sanctioned version of the tool from a trusted source. One thing you could try as a workaround is have a step in your ETA, just reread your reply...
Do you not have access to target/, or is it just that the plugin hasn't got as far as doing anything else if it cannot execute from ~/.m2? |
Beta Was this translation helpful? Give feedback.
-
|
Hi @ascopes.
That's why the property should be optional and, if it's not set, default to the current behaviour.
I was thinking to try the same yesterday. On the paper it should potentially work, but it would be nice to have a uniform build setup.
The plugin doesn't do anything else as soon as it can't run from |
Beta Was this translation helpful? Give feedback.
-
|
Would you be able to give that a go? If that works for you, it saves a fair amount of plumbing to work around this within the resolver integration. I would still be in the camp of suggesting that anything in m2 needs to be executable going forwards though, as many other components also make the same assumptions for other projects... it is a fairly standard pattern in that regards. From a security perspective the m2 still can be abused regardless of whether EXEs can reside in there or not. |
Beta Was this translation helpful? Give feedback.
-
|
In the mean time, I'll have a think about how this would need to look. |
Beta Was this translation helpful? Give feedback.
-
It doesn't seem to work. <?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<properties>
<java.version>21</java.version>
<grpc.version>1.75.0</grpc.version>
<protobuf-maven-plugin.version>3.8.1</protobuf-maven-plugin.version>
<protobuf.compiler.version>4.32.0</protobuf.compiler.version>
</properties>
<profiles>
<profile>
<id>windows</id>
<activation>
<os><family>windows</family></os>
</activation>
<properties>
<protobuf-download-directory>C:/Develop/protobuf</protobuf-download-directory>
</properties>
<build>
<pluginManagement>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-dependency-plugin</artifactId>
<executions>
<execution>
<id>download-protobuf</id>
<phase>initialize</phase>
<goals>
<goal>copy</goal>
</goals>
<configuration>
<outputDirectory>${protobuf-download-directory}</outputDirectory>
<artifactItems>
<artifactItem>
<groupId>com.google.protobuf</groupId>
<artifactId>protoc</artifactId>
<version>${protobuf.compiler.version}</version>
<type>exe</type>
<classifier>windows-x86_64</classifier>
</artifactItem>
<artifactItem>
<groupId>io.grpc</groupId>
<artifactId>protoc-gen-grpc-java</artifactId>
<version>${grpc.version}</version>
<type>exe</type>
<classifier>windows-x86_64</classifier>
</artifactItem>
</artifactItems>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>io.github.ascopes</groupId>
<artifactId>protobuf-maven-plugin</artifactId>
<version>${protobuf-maven-plugin.version}</version>
<configuration>
<binaryUrlPlugins>
<binaryUrlPlugin>
<options>@generated=omit</options>
<url>file:///${protobuf-download-directory}/protoc-gen-grpc-java-${grpc.version}-windows-x86_64.exe</url>
</binaryUrlPlugin>
<binaryUrlPlugin>
<options>@generated=omit</options>
<url>file:///${protobuf-download-directory}/protoc-${protobuf.compiler.version}-windows-x86_64.exe</url>
</binaryUrlPlugin>
</binaryUrlPlugins>
</configuration>
</plugin>
</plugins>
</pluginManagement>
</build>
</profile>
<profile>
<id>unix</id>
<activation>
<os><family>unix</family></os>
</activation>
<build>
<pluginManagement>
<plugins>
<plugin>
<groupId>io.github.ascopes</groupId>
<artifactId>protobuf-maven-plugin</artifactId>
<version>${protobuf-maven-plugin.version}</version>
<configuration>
<binaryMavenPlugins>
<binaryMavenPlugin>
<groupId>io.grpc</groupId>
<artifactId>protoc-gen-grpc-java</artifactId>
<version>${grpc.version}</version>
<options>@generated=omit</options>
</binaryMavenPlugin>
</binaryMavenPlugins>
</configuration>
</plugin>
</plugins>
</pluginManagement>
</build>
</profile>
</profiles>
<build>
<sourceDirectory>${project.basedir}/src/main/java</sourceDirectory>
<testSourceDirectory>${project.basedir}/src/test/java</testSourceDirectory>
<resources>
<resource>
<filtering>true</filtering>
<directory>src/main/resources</directory>
</resource>
</resources>
<testResources>
<testResource>
<filtering>true</filtering>
<directory>src/test/resources</directory>
</testResource>
</testResources>
<plugins>
<plugin>
<groupId>io.github.ascopes</groupId>
<artifactId>protobuf-maven-plugin</artifactId>
<executions>
<execution>
<id>generate</id>
<goals>
<goal>generate</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-dependency-plugin</artifactId>
</plugin>
</plugins>
</build>
</project>and below the output from Maven: |
Beta Was this translation helpful? Give feedback.
-
|
Ok, nevermind. I should have read https://ascopes.github.io/protobuf-maven-plugin/changing-protoc-versions.html first. |
Beta Was this translation helpful? Give feedback.
-
|
Ah so binary maven/url plugins are specifically for plugins that protoc hooks into, rather than protoc itself. Just saw your reply as I was responding. That is good to know. (was typing this as you replied) Just had a poke around the Aether documentation for this to see what is available. If I can control this from the Aether point this would be much more preferable than passing a bunch of config for this around to every site that resolves anything. Looks like Aether exposes a LocalRepositoryManager which exposes a LocalRepository. I could in theory do something to allow overriding this, which would then only apply to things that this plugin resolves, but due to the nature of when it gets initialized, it'd most likely only be able to be controlled via a JVM property that you place in # -*- .mvn/maven.config -*-
# Override the location artifacts are written to in order to
# work around group policy constraints for executable binaries.
-Dprotobuf.repository.path=./.pmp_cacheI don't know how that would work with loading the settings.xml from the existing .m2 though. I have a feeling this would possibly break how authentication works with your corporate Maven package proxy I assume you'd be using. I can also see some APIs for a concept called "workspace repositories" which are supposed to help support things like IDEs that integrate with the resolver. Maybe this could utilise that instead, as there is a point for integration on RepositorySystemSession for that as well. One caveat of this is that if this was possible, I am not sure if these APIs are only for resolver API v2 or not. Per GH-772 and GH-773, the v2 resolver is only compatible with Maven 4.0. If that is the case then this would be a no-go as I still need to retain compatibility with the v1 resolver APIs to keep supporting Maven 3.8 and 3.9. |
Beta Was this translation helpful? Give feedback.
-
|
That definitely isn't an ideal solution for you with regards to the boilerplate needed to achieve that. I'll have a play if I get time at the weekend and see if I can find a less cruddy way of doing this for you. I can't promise anything but I will see what I can do. Leave it with me. For now though, it sounds like you at least have a workaround. |
Beta Was this translation helpful? Give feedback.
-
|
What I can say from a consumer prospective is that there should be just a property to set with the path that it should be used similar to how the dependency plugin does and then set it only on the With that being said, an issue I discovered is that the |
Beta Was this translation helpful? Give feedback.
-
|
The ability to control this from a Maven POM-based property may not be possible due to where Maven configures the Aether stack from. Effectively by the time the goal is configured, the other components for resolution will already have been configured. Is there a reason you'd need a profile for this if it is putting files in a consistent place relative to the project directory in this case (e.g. in target/)? -- File URIs have to use forward slashes to be compliant; this is intentional: https://datatracker.ietf.org/doc/html/rfc8089#section-2. All URI parsing is handed off to the standard library to deal with, where it is then processed via the You should be able to use relative paths rather than hardcoding absolute paths: Using a relative path to a |
Beta Was this translation helpful? Give feedback.
-
Given that the dependency plugin is able to offer such capability, I don't see why it's impossible. There it is possible to change the output directory with no issues through a configuration property of the plugin.
Because, as said, for Widows I can use only a specific path ( |
Beta Was this translation helpful? Give feedback.
-
|
The dependency plugin is designed purely for this purpose, rather than being a small moving component in a large number of flows. This plugin doesn't build on top of the dependency plugin, rather it directly operates on Maven APIs directly. In the case of transitive dependencies, the entire tree would have to be relocated otherwise it opens you up to more issues. Are you not working withing C:\Develop when building the project? Otherwise other functionality is not going to work for you either such as JVM plugins since they rely on generated scripts exec'd on the OS level to bootstrap correctly. |
Beta Was this translation helpful? Give feedback.
-
|
I may be able to put an edge case in purely for this case but my concern is that if more usecases for this come along, it will be increasingly difficult to handle this once transitive requirements come along (which is the case for JVM-based protoc plugins). There might be an easy workaround, I'll know when I have time to look fully at the weekend some point, and shall get back to you. |
Beta Was this translation helpful? Give feedback.
-
Than everyone needs to change their settings only for a single project. That's not worth the effort. Regarding that folder, executables can run if placed in there as soon as they are 'plain' ones (e.g.: no installers) and don't use anything outside such folder. |
Beta Was this translation helpful? Give feedback.
-
|
In this case, it feels like you are likely to encounter other issues moving forwards. This plugin relies on the ability to execute things at certain paths to be able to integrate with protoc sensibly from the Java runtime. The issue here is that protoc itself needs the ability to call other things that may be native executables or packaged JARs. Protoc itself only allows executing system executables via the OS syscalls for fork/exec and whatever the Windows NT equivalent of that is. To support running JARs, Maven plugins for protoc have historically called other tooling to transpile the JARs into native binaries. This plugin removes the need for a C compiler toolchain by generating OS specific bootstrap scripts that set up the classpath, and can be invoked by protoc by exploiting the fact all major operating systems can invoke shell/batch scripts via fork/exec. This means any solution is going to also have to take this into account to be fully functional rather than a partially-baked workaround that will result in future bugs being raised. Whilst I could further look into this, I think without having a less restrictive environment, you are going to encounter more issues moving forwards. My suggestion for now would be to use protoc on the system path for development machines to reduce the boilerplate. You can then utilise a Maven profile for Windows to set Apologies I cannot provide a better solution right now... appreciate the pain it can be working on a corporate system locked down like this from my own experience in the past. It'd definitely be worth raising this kind of difficulty internally if possible though, since there are several other projects such as Maven Wrapper and Gradle Wrapper which will also be ill-equipped to handle this from what I can see. |
Beta Was this translation helpful? Give feedback.
-
|
Maven itself is also installed in That's why I personally think that just allowing to download those executables in a configurable path should be suffice. |
Beta Was this translation helpful? Give feedback.
-
|
Maven exec plugin will reside in m2 in this case; other tooling like mvnw will still reside outside this path though. |
Beta Was this translation helpful? Give feedback.
-
|
We don't use the wrapper. ;) |
Beta Was this translation helpful? Give feedback.
-
|
Since a workaround exists, going to transfer to a discussion so this can be further explored. This will also give a space for other use cases to be flagged up. My main concern relates to the process of bootstrapping JVM plugins in this case if such a constraint exists. If this is something you'd be willing to try, it would be very useful context. |
Beta Was this translation helpful? Give feedback.
-
|
Please can you run the following from your machine to verify this workaround works for you: $ git clone -b feature/gh-782-sanctioned-executable-paths https://github.com/ascopes/protobuf-maven-plugin
$ cd protobuf-maven-plugin
$ mvn clean install -Dmaven.test.skip -Dinvoker.skip -Dcheckstyle.skip -Dlicense.skipIn your project you are using this from, use Once I know this works on your configuration, I'll finish the unit tests, raise a pull request, and get this released for you. Thanks. |
Beta Was this translation helpful? Give feedback.
-
|
Will it really be a breaking change?
Nevertheless, I can try to test on Monday.
I just hope that the setup consists in just setting a user property in the
Windows profile, so that I don't have to duplicate the definition of the
plugin. :)
Il sab 30 ago 2025, 12:54 Ash ***@***.***> ha scritto:
… I've got a potential fix for this working locally that intercepts the
requested binary locations prior to running protoc in such a way that it
should cover *all* use cases.
image.png (view on web)
<https://github.com/user-attachments/assets/dd8a7b08-1480-4df4-836c-0092be4884db>
I'll push this to a branch, but would request that you build this branch
on your machine and run it so that I can verify the fix works as expected
for you (since I don't want to make breaking changes to what it is doing
once I have published it and other people may be using it).
Will respond with some instructions shortly.
—
Reply to this email directly, view it on GitHub
<#782 (reply in thread)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ACAZGWAVXES6LFG4KPPCQS33QF7HHAVCNFSM6AAAAACFGREBY6VHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTIMRWGIZDMOA>
.
You are receiving this because you authored the thread.Message ID:
<ascopes/protobuf-maven-plugin/repo-discussions/782/comments/14262268@
github.com>
|
Beta Was this translation helpful? Give feedback.
-
|
I can try to test that as well.
I'll need to revert some stuff (i.e.: the workaround discussed previously), but it should be doable.
I'm new to protobuf and gRPC, so I my starting point was what gets
generated by the Spring Boot initializer with the spring-grpc dependency. :)
Il sab 30 ago 2025, 12:49 Ash ***@***.***> ha scritto:
… Was looking for you to try to use a JVM-based protoc plugin within your
environment, such as those used in the examples for the docs at
https://ascopes.github.io/protobuf-maven-plugin/using-protoc-plugins.html#pure-java-plugins
to see if you see the same issue with generated bat/sh scripts.
—
Reply to this email directly, view it on GitHub
<#782 (reply in thread)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ACAZGWHGZ3MUCPGVCTELF233QF6U3AVCNFSM6AAAAACFGREBY6VHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTIMRWGIZDKNQ>
.
You are receiving this because you authored the thread.Message ID:
<ascopes/protobuf-maven-plugin/repo-discussions/782/comments/14262256@
github.com>
|
Beta Was this translation helpful? Give feedback.
-
|
Ok, that's what I was expecting.
Are there other instructions I should be aware of?
Also, which one is the branch?
Il sab 30 ago 2025, 13:44 Ash ***@***.***> ha scritto:
… Breaking change
Only if I have to change it once I've released it.
Setting a user property
It'll be a Maven property, so you can either;
- configure it in the plugin configuration itself
- configure it in your <properties> block
- configure it from a profile
- configure it in a parent POM you inherit from
- pass -Dprotobuf.sanctioned-executable-path=... to Maven on the
command line
- set -Dprotobuf.sanctioned-executable-path=... in .mvn/maven.config
- configure the MAVEN_ARGS (or MAVEN_OPTS, I forget which) environment
variable to -Dprotobuf.sanctioned-executable-path, which you can do in
your Windows policy.
—
Reply to this email directly, view it on GitHub
<#782 (reply in thread)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ACAZGWC27W57UEB3LCL76LL3QGFAPAVCNFSM6AAAAACFGREBY6VHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTIMRWGI2DGNA>
.
You are receiving this because you authored the thread.Message ID:
<ascopes/protobuf-maven-plugin/repo-discussions/782/comments/14262434@
github.com>
|
Beta Was this translation helpful? Give feedback.
-
|
Noted.
By the way, I would personally name the property something more generic
like `protobuf.binaries.output-directory` instead.
Someone may use it to put the binaries somewhere else for some reason (e.g.
to not pollute .m2).
Of course, mine is only a suggestion. ;)
Il sab 30 ago 2025, 14:08 Ash ***@***.***> ha scritto:
… $ git clone -b feature/gh-782-sanctioned-executable-paths https://github.com/ascopes/protobuf-maven-plugin
$ cd protobuf-maven-plugin
$ mvn clean install -Dmaven.test.skip -Dinvoker.skip -Dcheckstyle.skip -Dlicense.skip
so you want feature/gh-782-sanctioned-executable-paths and to run that
maven command
—
Reply to this email directly, view it on GitHub
<#782 (reply in thread)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ACAZGWDADIGL635UQ7S5ETL3QGH5LAVCNFSM6AAAAACFGREBY6VHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTIMRWGI2TCNQ>
.
You are receiving this because you authored the thread.Message ID:
<ascopes/protobuf-maven-plugin/repo-discussions/782/comments/14262516@
github.com>
|
Beta Was this translation helpful? Give feedback.
-
|
As a separate note, just spotted https://github.com/ascopes/protobuf-maven-plugin/actions/runs/17354518668/job/49265549171?pr=783 is apparently falling over just on Windows machines so I'll have to try and investigate further today and get back to you once I have sussed out what is failing there (I don't have a Windows environment to test in locally so will probably have to spin a VM up to try it). |
Beta Was this translation helpful? Give feedback.
-
|
Releasing as 3.9.0 to Maven Central |
Beta Was this translation helpful? Give feedback.
-
|
@cdprete hey... totally random question for you. Your work machine using this configuration. Does it get spooked by symbolic links in your E.g. suppose I do the following from Git Bash: Does this succeed? Asking since I am curious as to whether OS level juntions/symbolic links are a more effective workaround to actually copying binary blobs around. I highly doubt this will work but it has been a thought experiment that has been troubling me a bit recently. I don't have a dev setup on Windows to try this out with unfortunately. |
Beta Was this translation helpful? Give feedback.

Uh oh!
There was an error while loading. Please reload this page.
-
Brief description
Hello.
In the company where I work, we're not allowed to run Windows executables outside of a specific directory.
Therefore, with the current settings the plugin offers, the 2 executables which get downloaded get placed under the repository plugin folder of Maven which is an allowed directory from where to start the executables.
Of course, I think (I need to test this), that I may workaround this by setting the
localRepositoryin thesettings.xmlof Maven to such allowed directory, but then this would impact all my projects and it isn't very nice.Ideally an optional property should allow to set such output/working directory.
Steps to reproduce
See the description above.
Error message
Logs
Operating system
Windows 10 22H2
JDK version
21
Maven version
3.9.9
Plugin version
3.8.1
Protoc version
4.32.0
Additional details
No response
Would you like to contribute a bugfix?
Beta Was this translation helpful? Give feedback.
All reactions