404 from @latest and @v/list when pre-filling the disk cache

[brian73286]

Describe the bug
I get 404 from @v/list and @latest endpoints for modules when using athens on a computer that has never cached internet resources, after following instructions for pre-filling the disk cache.

Error Message
When I GET $HOST:$PORT/github.com/julienschmidt/httprouter/@v/list my client receives 404 and kubectl logs <POD_NAME> shows:

time="2019-10-09T12:34:56Z" level=info msg="exit status 1: go list -m: github.com/julienschmidt/httprouter@latest: git ls-remote -q origin in /tmp/athens<DECIMAL>/pkg/mod/cache/vcs/<HEXADECIMAL>: exit status 128:\n\tfatal: unable to access 'https://github.com/julienschmidt/httprouter/': Could not resolve host: github.com\n" http-method=GET http-path=/github.com/julienschmidt/httprouter/@v/list http-url=/github.com/julienschmidt/httprouter/@v/list kind="Not Found" module= operation=download.ListHandler ops="[download.ListHandler pool.List protocol.List vcsLister.List]" version=

When I GET $HOST:$PORT/github.com/julienschmidt/httprouter/@latest my client receives 404 and kubectl logs <POD_NAME> shows:

time="2019-10-09T12:34:56Z" level=info msg="exit status 1: go list -m: github.com/julienschmidt/httprouter@latest: git ls-remote -q origin in /tmp/athens<DECIMAL>/pkg/mod/cache/vcs/<HEXADECIMAL>: exit status 128:\n\tfatal: unable to access 'https://github.com/julienschmidt/httprouter/': Could not resolve host: github.com\n" http-method=GET http-path=/github.com/julienschmidt/httprouter/@latest http-url=/github.com/julienschmidt/httprouter/@latest kind="Not Found" module= operation=download.LatestHandler ops="[download.LatestHandler pool.Latest protocol.Latest vcsLister.List]" version=

To Reproduce
Using a computer that has never connected to the internet, follow the instructions at #1414 as they were on 2019-10-08 using the v0.2.0 tag of github.com/plexsystems/pacmod to pack up the v1.3.0 tag of github.com/julienschmidt/httprouter for athens. That creates 3 files: go.mod, v1.3.0.info, v1.3.0.zip.
Rename v1.3.0.zip to source.zip.
Using kubernetes run the image gomods/athens:v0.6.1 with persistent storage mounted at /opt/athens/storage and overwrite the image's included config by mounting yours at /config/config.toml.
Your config.toml contains:

StorageType = "disk"
[Storage]
    [Storage.Disk]
        RootPath = "/opt/athens/storage"

using kubectl cp copy the 3 files, one of was renamed, to be under /opt/athens/storage/github.com/julienschmidt/httprouter/v1.3.0/ and see they default to 0640 file permissions.
It doesn't seem to make a different but restart the pod for good measure in case the files need to exist in persistent storage before the pod launches.

Expected behavior
I expect output similar to what is documented at https://docs.gomods.io/intro/protocol/

Environment (please complete the following information):

• OS: Kubernetes v1.14.5 runs the image, client is CentOS 6.10
• Go version : 1.13
• Proxy version : gomods/athens:v0.6.1
• Storage (fs/mongodb/s3 etc.) : kubernetes persistentVolumeClaim

Additional context
These work perfectly fine:
GET $HOST:$PORT/github.com/julienschmidt/httprouter/@v/v1.3.0.info returns valid JSON with sane values
GET $HOST:$PORT/github.com/julienschmidt/httprouter/@v/v1.3.0.mod returns text that matches the contents of the module's go.mod file even though the Content-Type is application/json
GET $HOST:$PORT/github.com/julienschmidt/httprouter/@v/v1.3.0.zip serves the zip archieve for download and when downloaded and extracted the files & directory hierarchy matches the module.

This entire issue was hand-typed so feel free to ask if anything in the error messages is a typo.

[brian73286]

I'm guessing either athens is failing to do some sort of logic or I need my config.toml to specify GOPRIVATE, GOPROXY, and/or GONOPROXY in athen's GoBinaryEnvVars property. If it is the latter then that should also be documented.

[jpreese]

I've made a couple comments on PR #1414 relating to the guide, and my gut says it's because of the step that has you renaming from v1.3.0 to source.

When you run go mod download, it returns a .zip that is named the same as the version.

[brian73286]

I just tried renaming /opt/athens/storage/github.com/julienschmidt/httprouter/v1.3.0/source.zip to /opt/athens/storage/github.com/julienschmidt/httprouter/v1.3.0/v1.3.0.zip. The @v/list and @latest endpoints continue to return 404 and now GET $HOST:$PORT/github.com/julienschmidt/httprouter/@v/v1.3.0.zip also returns 404 instead of offering a file for download. I've reverted this rename.

[jpreese]

Interesting. I'll keep an eye on this thread, because if its supposed to be source, then I'd rather pacmod generate a file called source.zip so users aren't stuck renaming it.

I was just going off of the fact that when you use the Go toolchain, it tosses a v1.3.0.zip file in your mod cache.

[arschles]

if its supposed to be source, then I'd rather pacmod generate a file called source.zip so users aren't stuck renaming it.

@jpreese iirc naming it source.zip was an arbitrary decision. We can of course have Athens look for a $VERSION.zip on the backend, but we'll need to retain backward compatibility too, so it doesn't sound worth it to me.

On the other hand, Instead of changing pacmod completely, what do you think about adding a new command called pacmod import-athens or something similar? That comment could do the same thing as packmod pack and then rename & move the files into the disk storage root.

[jpreese]

I'm not opposed to that. I just wanted to make sure we weren't missing anything with the naming of source.zip.

Because the go mod toolchain uses $VERSION.zip, I think it would be worthwhile for Athens to do the same, if even at a major version bump.

[arschles]

if even at a major version bump

We haven't done one of these. We haven't made the plunge to a v1.0.0 yet either 😉

That said, would you be up for submitting a PR that does the backward compatible $VERSION.zip support in the disk driver? No pressure if not - I'll pick it up.

Also, if you're open to it, I'm totally open to adding a pacmod import-athens command. Let me know.

[jpreese]

I can tackle that @arschles.

For pacmod, my gut would be to add an output / name sort of parameter that would allow the user to name their artifacts. I think that gives us what we need without needing to create a special purpose command for Athens.

Though, if we update Athens to support both $VERSION and source (for backwards shenanigans), I don't think we'd need it, right? We'd just let pacmod continue to archive modules as $VERSION.zip, and Athens would be able to handle it.

[brian73286]

I just verified this issue also affects the new v0.7.0 release in the same ways.

I tried copying source.zip to v1.3.0.zip, then restarting the container so it is running with four files under /opt/athens/storage/github.com/julienschmidt/httprouter/v1.3.0/

• go.mod
• v1.3.0.info
• v1.3.0.zip
• source.zip.

Where the last two are copies of each other. The hope was that instead of providing a file named one way or the other, try giving it both. This did not fix the 404's I get for /@v/list and /@latest. I think the problem may be deeper than the naming of that file.

[jpreese]

I don't seem to be having the same issues that you're running into, @brian73286. The list command should be working fine as that just looks at the folders. The name of the artifacts shouldn't matter.

This was how I was spinning up the container:

docker run -e "ATHENS_STORAGE_TYPE=disk" -e "ATHENS_DISK_STORAGE_ROOT=tmp" --rm -it -v C:\Users\jpree\Desktop\athens:/tmp -p 3000:3000 gomods/athens

Are your storage type and storage path vars set?

[jpreese]

@arschles I took some time today to look into some of the internals of how Athens handles different filesystems, and I don't necessarily see a problem with how Athens is storing the modules.

I think it's a little confusing that it doesn't mimic the same file structure as the Go tooling, but from an API point of view, the file structure shouldnt/doesn't matter. I was able to spin up a local Athens proxy and run the commands fine, including having my archive named source.zip.

[arschles]

@brian73286 I haven't been active in this issue for some time, sorry about that. If the issue is still happening, let me know and I'll help debug some more.

[kmalloc]

I am experiencing same issue, and the reproducing step is exactly the same as described in this thread.

following works fine:

GET $HOST:$PORT/github.com/kmalloc/l5/@v/v1.0.0.info
GET $HOST:$PORT/github.com/kmalloc/l5/@v/v1.0.0.mod
GET $HOST:$PORT/github.com/kmalloc/l5/@v/v1.0.0.zip

however, @latest and @v/list return 404.

athens I am using is 0.7.2. go 1.12.5

[arschles]

@brian73286 @jpreese @kmalloc I couldn't reproduce this again on v0.7.2. Here is how I ran Athens:

ecomaz :: ~/Desktop/athens-volume » docker run -v $ATHENS_STORAGE:/var/lib/athens \ 
   -e ATHENS_DISK_STORAGE_ROOT=/var/lib/athens \
   -e ATHENS_STORAGE_TYPE=disk \
   --name athens-proxy \
   --restart always \ 
   -p 3000:3000 \
   gomods/athens:v0.7.2

I did an export ATHENS_STORAGE=$PWD before I ran this. Forgot to copy that in above.

That command runs Athens with the default config.toml file, so it's using disk storage and the directory is on my host.

Here are my logs for what I did:

ecomaz :: ~/Desktop/athens-volume » curl localhost:3000/github.com/julienschmidt/httprouter/@latest      
{"Version":"v1.3.0","Time":"2019-09-29T23:21:22Z"}
ecomaz :: ~/Desktop/athens-volume » curl localhost:3000/github.com/julienschmidt/httprouter/@list  
404 page not found
ecomaz :: ~/Desktop/athens-volume » curl localhost:3000/github.com/julienschmidt/httprouter/@v/list
v1.0.0
v1.1.0
v1.2.0
v1.3.0%                                                                                                                          
ecomaz :: ~/Desktop/athens-volume » curl localhost:3000/github.com/julienschmidt/httprouter/@v/v1.3.0.mod
module github.com/julienschmidt/httprouter

go 1.7
ecomaz :: ~/Desktop/athens-volume » ls -al
total 0
drwxr-xr-x   3 aaronschlesinger  staff   96 Mar 20 10:32 .
drwx------@ 26 aaronschlesinger  staff  832 Mar 20 10:23 ..
drwxr-xr-x@  3 aaronschlesinger  staff   96 Mar 20 10:32 github.com

After these commands, Athens has downloaded the module into my $PWD/athens-volume directory. It looks like this:

ecomaz :: ~/Desktop/athens-volume » ls github.com/julienschmidt/httprouter/v1.3.0/                                         127 ↵
go.mod      source.zip  v1.3.0.info

So it's certainly important that the source code is called source.zip. Athens won't work with it being v1.3.0.zip. Same goes for go.mod instead of v1.3.0.mod.

@kmalloc I think that the issue you mentioned is a bit different because github.com/kmalloc/l5 doesn't seem to exist. Did you load that code into your Athens storage using pacmod?

Let me know if you'd like logs for Athens, happy to share them.

Finally, I tried the docs at https://docs.gomods.io/configuration/prefill-disk-cache/ and noticed some things are off:

• pacmod pack $VERSION . doesn't work. It needs to be pacmod pack $MODULE_NAME $VERSION. I'll fix that in a PR shortly
• Something is off either on permissions or the file structure. I suspect it's permissions because the directory structure in the docs looks the same as what Athens shows. I need to look into more detail

Let me know if you have any ideas regarding the second bullet